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ABSTRACT 
We derive and examine the Neyman-Pearson Detection of an FSK modulated waveform in white 
Gaussian noise. We specifically examine three cases: 1) detection when the waveform is known 
exactly; 2) detection when the waveform is known down to an unknown initial phase and; 3) 
detection of the waveform when the energy level is known but the signal is not. We then derive 
robust detection schemes where the noise comes from a broader class of stochastic processes 
and includes the possible occurrence of outlier data. We compare these robust detection 
schemes to their non-robust equivalents under various noise models. 
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1 Introduction 
1.1 Overview 

Signal Detection is a key step in many communication, RADAR, and signal geolocation systems. 
Optimal signal detection design is dependent on the apriori knowledge of the signal being 
detected as well as the correct model for the transmission or noise channel. For any real world 
application it is extremely difficult to precisely model the noise channel. Furthermore, optimal 
signal detectors' overall performance can be extremely sensitive to the noise channel. In 
contrast, robust detectors provide good performance at the assumed noise channel and 
maintain good performance over a wide class of possible noise channels. This thesis examines 
non-robust and robust detection schemes for three cases of apriori signal knowledge and 
provides detailed analysis comparing these detection schemes. 

In Chapter 2 we derive and compare optimal signal detection of a Frequency Shift Keying (FSK) 
modulated signal in an additive white Gaussian noise channel for three typical cases of apriori 
signal knowledge. The relative performance gain between the cases is found to be significantly 
different than approximate performance gain figures derived using first and second order 
statistics only. 

In Chapter 3 we derive robust signal detection schemes for the three cases examined in Chapter 
2. For two of the three cases, the robust detection schemes derived in Chapter 3 can be shown 
to be optimal; they are Neyman Pearson detection schemes designed at the least favorable 
density pair. For the remaining case, an ad-hoc but intuitively pleasing detection scheme is 
designed and shown to provide good performance in the presence of low frequency outliers. All 
cases are compared to the non-robust detection schemes found in Chapter 2 under various 
noise models. 



1.2 Notation and Symbols 

Throughout this thesis all symbols used to represent variables, random variables, constants, or 
functions are defined when first introduced. The table below contains a brief description of 
terms which are common among multiple chapters and sections of this thesis. The table is not 
an exhaustive list, as many symbols used for a given section are unique to that section and are 
therefore best introduced when they are first used. Furthermore, the table does not contain a 
precise definition of all terms listed as such a precise definition can only be provided in the 
context of the equations in which the terms are used. The table is placed here to introduce the 
reader to common symbols and enable the reader to understand a given chapter or section 
without reading all preceding text. 



Table 1: Table of mathematical symbols. 



E 


Energy of the signal per sample 


fi 


Probability Density function assuming the i th hypothesis is active 


Pi 


Cumulative Density function assuming the i th hypothesis is active 


3 t 


The class of all possible density functions active if the i th 
hypothesis is active 


Hi 


The i th hypothesis 


/oW 


The zeroth ordered Bessel function evaluated at x 


n 


The number of samples 


No 


The noise spectral density 


Pfa 


Probability of false alarm 


Pd 


The power or probability of detection 



Table 1 (con't.) 



Q n (a,b) 


The Marcum-Q function of n th order evaluated at (a, b) 


x t 


The real part of a complex sample from the observation vector 


Vi 


The imaginary part of a complex sample from the observation 
vector 


a 


False alarm constraint 


y 


Threshold value used in detection tests 


s 


The decision rule for a given detection test 


£i 


The frequency of outliers assuming the i th hypothesis is active 


o 


The initial phase offset of the signal 


n 


The mean of a random variable 


p 


The signal-to-noise ratio 


a 


The standard deviation of a random variable generally Gaussian 


V 


The standard deviation of a random variable 


<Kx) 


The probability density function of a zero mean, unit variance 
Gaussian variable evaluated at x 


O(x) 


The cumulative density function of a zero mean, unit variance 
Gaussian variable evaluated at x 



2 Parametric Detection 

2.1 Introduction 

After the signal is brought to baseband, it is low-pass filtered and sampled. We then obtain the 
following two data sample vectors 

X(i)= A(0+N t (i) (Eq. 2.1.1) 

Y(Q = B(Q + N 2 (Q (Eq. 2.1.2) 

N t (i) and N 2 (Q are Independent White Gaussian Noise processes with a 2 = N . The 
expressions fon4(£), 6(0 depend on the low-pass filter used as shown in [1]. To simplify our 
presentation, we assume the deployment of an ideal low-pass filter without distortion such that 

A(Q = VI*cos(0 o + P(O) (Eq. 2.1.3) 

B(i) = VI*sin(0 o + P(O) (Eq. 2.1.4) 

where G is a uniform random variable on the interval [— n, n]; for scenarios where it is 
unknown and a fixed constant for scenarios where it is known. P(i) is in general a random 
variable; for the case where the waveform is unknown and a deterministic function; for the case 
where the waveform is known. Finally, E is the energy of the signal. 

2.2 Detection when the waveform and initial phase are known 

In this case, P(i) is no longer random; equivalently, Prob{P(i) = p(i)} = 1< for some 
deterministic p(i). The optimal detector is given by deciding H 1 if 



n n 

- Y x(i)cos {6 + p(0) + - Y y(i)sin (9 + p(i)) > y. (Eq. 2.2.1) 

ni—i ni—i 

i = l i = l 



Let 



n n 

Z = - Y x(i)cos (G + p(O) + - Y yCOsin (0 O + p(i)) (Eq. 2.2.2) 

i = l i = l 



The detection statistics are then, 



nz\H 1 ) = M-==) (Eq. 2.2.3) 



nz\H ) = ^[-^=\ (Eq. 2.2.4) 



P fa = Prob(Z > y\H ) = 1 - <*>( ^=) = <p( - ^ J = a (Eq. 2.2.5) 



y = -jN /n (a) (Eq. 2.2.6) 



fyVn — yfnE\ (^fnE — yyfn\ 
P d = ProbiZ > Y \HO = 1 - 0[ - = = 0[ j=A (Eq. 2.2.7) 

V jn / v v^ / 



NnE * \ 

P d = 0\^=+0 1 (a)) (Eq. 2.2.8) 



Define p = — . And re-express the probability of correct signal detection as follows: 



P d = 0{j2np+ 0~ 1 (aj) (Eq. 2.2.9) 



2.3 Detection when the waveform is unknown (energy detection) 

P(i) is now a random variable, which in general depends on the FSK index. If we do not know 
the FSK index, we will work under the model that each P(i) is a random variable uniformly 
distributed on [— n, n]. Our optimal detector is now a simpler energy detector given by deciding 
H t \f 



-Tixf+yf] >Y (Eq. 2.3.1) 

rti—t 

i = l 



Our detection statistics are 



n (Zn\ n 1 / Zn\ 

^"^ z-O-D^ U e A-w) (Eq ' 2 - 3 - 21 

n_i 

n /—Zn — nE\/Z\2 2 f\n 2 EZ\ , r ., -, -,> 

P, a = ProHZ>y\H,)= I (i z_ 5 _ 5 _(_) exp (-—) = « (Eq. 2.3.4) 



Let x = Zn/iV 



vn/Af dX 2H^T)! (x) "" leXP (-2) = a < E * 2 " 3 - 5 ) 



Define 



F " (X) = { dT 2"(n-l)! (T) "" leXP ("D (Eq - 23 - 6) 

F„(x) is the cumulative distribution function of the gamma random variable with parameters 2 
and n. 



(ny\ 



Pfa = 1 " £ 



7 = — Fn-^l-a) (Eq. 2.3.8) 

n 



P d = ProKZ > yWD = Q n ( & fe ) (Eq. 2.3,9) 



Let p = — . Then, 



^d = Qn U^,^F-\l-a)\ (Eq. 2.3.10) 



2.4 Detection when the waveform is known down to an unknown initial phase 

In this case P(i) is no longer random. Equivalently, there exists a deterministic p(i) such that 
Prob{P(i) = p(i)} = 1- We will design the Neyman-Pearson Detector for such a model. The 
Neyman-Pearson Detector decides H 1 if 

f rf' yn Ur (Eq. 2.4.1) 

while H is decided otherwise, where x n , y n are the observed data from the processes 
X(n), Y(ri), where / Hi (x n ,y n ), is the probability density functions when H t is active, and where 
y is a threshold determined by some false alarm constraint a. We will show that / Hi (x",y") 
does exist for both i = 0,1. To find fni(x n , y n ), we note by the theorem of total probability: 



1 f n 



/ni(l",y")=r: f lll (.x",y"\e = B)de (Eq. 2.4.2) 



And 



IL IL 

f H1 (x n ,y n \G = 6) = Y\fHi(.x t ,y t \e = 6) = Y\fHi(.x t \$„ = B)f H1 (yi\9 

i=l i=l 

(Eq. 2.4.3) 

n ij- y[E *cos(0 + p(Ol .J'i- V£*sin(0 + p(i))_ 

= Q) = \ <K F= ) ^( F= ) 

1 1 -ML ./AL 



• =1 ,-.„ ./No 



where ^(x) is the probability density function for the zero-mean unit Gaussian random variable. 

f H i(.x n ,y n ) = ^j f H1 (x n ,y n \ e = 6)d6 = (Eq. 2.4.4) 



1 1 1 
2nN, 



^T^r^l exp(- — Y(x ; - V£cos(0 + p(O)) A 2 + 
o (2H)n.L 7r 2JV £ (Eq. 2.4.5) 

(y ; - VIsin(0 + p(i))) A 2 ) rf6> = 



n 

c * exp (- — (^ [xf + y f] + nE) ) * 

£=1 

exp (— ) [x t cos( + p(0) +y ; sin( + p(i))] )d0 



(Eq. 2.4.6) 



i i 



where we have defined, c = — =■- — — — . In contrast to the detection case where both the 

W™ (27T) n+1 

waveform and the phase offset are known, in this case, where the waveform is known but not 
the phase offset, /Hi(x",y") is not memoryless, since 



f H i(.x n ,y n ) = ^j f H1 (.x n ,y n \ e = 6)d6 * 



f H i{x n ,y n )^j f H1 {x n -\y n - 1 \o = e)de 



= f H i(x n ,y n )///i(^ n - 1 ,y n - 1 ) 



To ultimately simplify the detection scheme, we now seek to simplify the expression for 
fm(x n ,y n ). We will start with the integrand 



ex P (tT / fc cos ( 6 + P W) +y; sin( + p(0)] ) (Eq. 2.4.8) 



Let )Uj = tan 1 ( — ). The integrand above can be rewritten as 

i — n 

exp(— ^[[^f+y^McosCHi) cos(0+p(O) +sinQx £ ) sin( + p(i))}] ) = (Eq. 2.4.9) 



i = l 



exp 



(a^Z^ 2 + y2 ^ cos (p(0 + 9 " ^ ) = (Eq ' 2 ' 4 ' 10) 



exp(-^^[xf+yfpi?e{exp(jp(i)+je-JHi)} j= (Eq. 2.4.11) 

exp Uei-j^^Jxf + y?Y exp (jp(0 + jO - ju ; )} J = (Eq. 2.4.12) 



exp Ue{exp (/0) ^Jj X ' ? + y ^ exp (jp(0 " j ^ )} I (Eq - 2A13) 



i = l 



where i?e{x} denotes the real part. Let 



where s* denotes the conjugate of s. Finally, define ay such that 



With these substitutions we have 



The overall integral we wish to evaluate is 



Finally, with the change of variables = ay + 9 , we write, 
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h(i) = exp 0'p(0) 



(Eq. 2.4.14) 



Ti = [xf + yf] 2 exp Q'mO 



(Eq. 2.4.15) 



i = l 



(Eq. 2.4.16) 



V = 



1L 

£=1 



(Eq. 2.4.17) 



1L 

i = l 



jw 



(Eq. 2.4.18) 



i — n 

-jj-}J. x ? + yf] 2 ex P OpCO - m) 



i = l 



No 



Ve io> (Eq. 2.4.19) 



///i(* n ,y n ) = c*exp(- — [T 2 

zyv 



f n VI 

+ nE]) exp (—V * Re{exp(j[a) + 9])} )d9 

J-n N Q 



(Eq. 2.4.20) 
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-| rTl + d Fn 

c * exp (- — [T 2 + nE]) exp (—V * Re{exp(jr)} )dt = (Eq. 2.4.21) 

2jV J-n+e "o 



i r VI 

c * exp (— — — [T + nE]) exp (— — V * cos (r))dr = 
2N J_ n N 



(Eq. 2.4.22) 



i r V£ 

2c*exp(- — [T z +n£])J exp (— K * cos ( T ))dr (Eq. 2.4.23) 



Using the identity 



1 r 71 

— exp (rcos (t))cLt = I (r) 

nJo 



(Eq. 2.4.24) 



We rewrite (Eq. 2.4.23) as, 



f H i(x n ,y n ) = c'exp (-^L[T 2 +nE])I (-^P) 



(Eq. 2.4.25) 



i i 
where c = 27rc = -=- — — . The likelihood ratio is thus 

W? (2tt)" 






(Eq. 2.4.26) 



Taking the log ratio and arranging terms, our test decides H t if 



log 



,VE 



>y' 



(Eq. 2.4.27) 



So far our test takes the observed sequence T t = x t + jy t , and the known transmitted signal 
(with unit amplitude) h(i), and computes 
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V = 



IV 

i = l 



(Eq. 2.4.28) 



Then, our test computes log \l ( — V) , and compares this test statistic to some threshold /' 
selected to attain probability of false alarm equal to a. However, since the function 
log \l f — V)\ is strictly a function of V and is monotonically increasing with V, for V > 0, while 
V is by definition always nonnegative, the detection rule 



log 



«■'£' 



> 



y(a) 



(Eq. 2.4.29) 



reduces to the equivalent inequality, 



V > y"(a) 



(Eq. 2.4.30) 



In other words, V is the sufficient statistics of the detection scheme. Finally, we can multiply 
both sides of the detection inequity by -V (we do this to more easily derived the analytic 
expression for the probability of detection). In conclusion, our detection rule decides H 1 if 



1 

-V 2 > y"{a) 
n 



(Eq. 2.4.31) 



where 



V = 






(Eq. 2.4.32) 



i = l 



and 
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T t = x t + jy t (Eq. 2.4.33) 

h(i) = exp(/p(0) (Eq. 2.4.34) 

A similar derivation for the Bayesian Detection of several orthogonal FSK signals is found in [2]. 
To evaluate the performance of our detection scheme, we now seek an analytic expression for 
the probability density function of-7 2 under both hypotheses. Define Z = -V 2 to be the 
random variable we seek to find under both H and H-j_. We first examine the statistics of 



it 

Y^Ti h*(Q (Eq. 2.4.35) 



i = l 



which is a linear combination of complex, independent Gaussian variables. Since any linear 
combination of Gaussian variables is Gaussian, the 



it 

Vr ; h*(Q (Eq. 2.4.36) 



i = l 



is a complex Gaussian variable and can be completely described by its mean and variance 
E V T t /i*(O| = m/E(cos0 o +;sin0 o ) (under H t ) (Eq. 2.4.37) 

£ Ytj/i'O) =0 (undertf ) (Eq. 2.4.38) 

VariyT i h*(.0\ = nN +jnN (under both H t and H ) (Eq. 2.4.39) 
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Therefore under H the distribution of Z is chi — square, and under H 1 , the distribution of Z is 
generalized chi — square. Specifically 



f {zw -^wr^{-w) < E «- 2A40 > 



, , x 1 /-Z-nE\ NnEZ\ 

«Z| Hl )=— exp(^-),„(— ) (Eq. 2.4.411 

Probability False alarm = Prob{Z > y\H } = 1 - J f{Z\H ) dZ = (Eq. 2.4.42) 

1 - I 1 - GXP (-2^)) = 6XP (- 2^) = a ^ 2A43 > 

Probability Detection = Prob{ Z > y\H t } = 1 - J /(Zl^) dZ = (Eq. 2.4.44) 

[°° 1 /-Z + nE\ (JnEZ\ 

I wM-^K-)'°\-K-) dZ (Eq ' 2 ' 4 ' 451 

Let x = JZ/N 

f°° f x 2 nE\ I \nE\ \n~E~ [y~\ 

L*H-t-wJ'° [*kr = Hk\k < E <- 2 - 4 - 46 ' 



where Qi(a, ft) is the Marcum-Q-function with M = 1. Define p to be the signal power to noise 
power ratio, per complex sample, where p = — . The probability of detection can be written as 

a function of the number of samples, the signal to noise ratio p and the probability of false 
alarm a, as: 
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P d = Q 1 (Jh^,j2\n(l/a)) 



(Eq. 2.4.47) 



Figure 2-1 shows the probability of detection versus the product np for various false alarms. 
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Figure 2-1: Power curves as a function of np for varies false alarm rates a. 
2.5 Comparison of the Detection Performance 

In this section, we will compare the detection performance for all three cases examined in 
sections 2.2, 2.3, and 2.4, by examining the induced probabilities of detection, as functions of 
a,p, and n. However, we will first compare the detection performance of Neyman-Pearson-like 
detectors which use just the mean and variance of our test statistic under H and H 1 , for all 
three cases. This analysis, being based on first and second order statistics, will lead to simpler 
expressions regarding comparisons between the three cases; however, the latter expressions 
are simply wrong when used for evaluation of the Optimal Neyman-Pearson detectors. The 
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reason we present the performance of detectors that use just the mean and the variance is to 
demonstrate the danger involved in limiting performance evaluations to just first and second 
order statistics. 

2.5.1 Detection Performance of Neyman Pearson-like Detectors 

Let [i m and a Hi be the mean and standard deviation of our test statistic under the acting 
hypothesis H it where i can be either or 1. Let us design our detection threshold to be 
Y = /%o + m(T Ho> where m is a constant. In other words, we set our threshold to be mo away 
from the mean, /i, under H . Clearly, the greater the selected m, the lower the resulting 
probability of false alarm. Furthermore, to simplify our detection performance analysis we will 
examine the values of n, and p needed for the case [i H1 = y in each of the three cases. We are 
thus using the first and second order statistics of our test statistic under H and just the mean 
under H 1 to compare the three cases. 

From section 2.2, in the case where the waveform as well as the initial phase are known, we 
have 

fi H0 = (Eq. 2.5.1) 



oho = jN /n (Eq. 2.5.2) 



y-Hi 



= VE (Eq. 2.5.3) 



Our detection comparison occurs when fi H1 = fi H0 + ma H0 or 

nE 



N =m (Eq. 2.5.4) 
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or equivalently when 



2np = m. (Eq. 2.5.5) 



From section 2.3 Detection when the waveform is unknown (energy detection), we have 



/%o = 2N (Eq. 2.5.6) 



2N, 



o 



o H o = —j= ( E q- 2 - 5 - 7 ) 



fi H1 = 2N +E (Eq. 2.5.8) 



Our detection comparison occurs when [i H1 = [i H0 + ma H0 or 



E ,_ 

yjn = m (Eq. 2.5.9) 



2N 



or equivalently when 

pyfn = m (Eq. 2.5.10) 

From section 2.4, Detection when the waveform is known down to an unknown initial phase, we 
have 

Mho = 2N (Eq. 2.5.11) 

a H0 = 2N (Eq. 2.5.12) 
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fi H1 = 2N +nE (Eq. 2.5.13) 

Our detection comparison occurs when [i H1 = [i H0 + ma H0 or 

pn = m (Eq. 2.5.14) 

To compare the three cases, let (ji CO h(p,ri),(x> energy (p,ri) and Wi„ CO (,(p,n) equal the 
expressions on the left hand side of (Eq. 2.5.5), (Eq. 2.5.10) and (Eq. 2.5.14), respectively. We 
then obtain the following ratios, as comparison quantities between the three tests 



te C oh(p, n ) _ 

= 2V^ (Eq. 2.5.15) 



^energy \P> fl) 

frW(p,n) 

(ti incoh(V> n ) 
^energy vP< 71 J 



= 2 (Eq. 2.5.16) 



= Vn (Eq. 2.5.17) 



Based on first and second order statistics, our analysis results in simple and easy-to-remember 
detection comparisons. From (Eq. 2.5.16) we conclude that the coherent detector performs 
approximately 3 dB better than the incoherent detector (in either p or n). From (Eq. 2.5.17) we 
conclude that the energy detector suffers a performance loss equal to ^Jn , when compared to 
the incoherent detector. As shown in the next subsection, these conclusions are not even good 
approximations of performance comparisons between the optimal Neyman-Pearson Detection 
schemes. 
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2.5.2 Detection Performance of the Neyman-Pearson Detectors 

We can compare the performance of the Neyman-Pearson Detectors via the use of numerical 
methods which evaluate the probability of detection p d , as a function of the variables a, p, and 
n. We first note that the incoherent and coherent detectors' performance can be expressed as a 
function of a and the product pn. Let y = pnand let us then define y C oh(Pd>Pfa) an d 
Yincoh(Pd>Pfa) as the smallest y, which achieves the probability of detection p d while 
maintaining the pf a , respectively for the coherent and incoherent Neyman-Pearson Detectors. 

The ratio Yincoh{Pd>Pfa)/Ycoh{Pd>Pfa) is similar to (Eq. 2.5.16) and represents the 
performance gain from knowing the initial phase offset, as compared to the initial phase offset 
being completely unknown. The energy detectors' performance depends on a, p, and n. Define 
Penergy(Pd>Pfa> n ) as the smallest p, which achieves probability of detection p d while 
maintaining the pf a , assuming that the number of samples in of the Neyman-Pearson Energy 
detector is n. Define n energy (p dl p fa ,p) and n incoh (p d ,p fa , p) similarly. The ratios 
Penergy(Pd,Pfa,n)/y incoh (p d ,p fa ) and n energy (p dl p fai p)/n incoh (p dl p fai p) are analogous 
to (Eq. 2.5.17) and represent the performance gain between the incoherent and energy 
detection schemes. 

Figure 2-2 shows Yi n coh(Pd,Pfa)/Ycoh(Pd,Pfa)- From this figure it is clear that incoherent 
detection is within 1.5 dB the performance of coherent detection for a pf a < 0.001 , as 
compared at any reasonable probability of detection. This is in contrast to the 3 dB performance 
difference derived using just first and second order statistics. 

Figure 2-3, Figure 2-4, and Figure 2-5 show Penergy(Pd,Pfa,n)/y iriC oh(Pd,Pfa) for n = 10, 100, 
and 1000 respectively. From these figures it is clear that energy detection performs significantly 
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better than the ^Jn performance loss (in terms of signal-to-noise ratio) derived from just first 
and second order statistics at all the pf a and p d values covered in these figures. 

Finally, Figure 2-6, Figure 2-7, and Figure 2-8 show n energy (p d ,p fa ,p)/n incoh (p d ,p fa ,p) for 
Pf a = le — 1, le — 3 and le — 5 respectively. From these figures it is clear that energy 
detection performs significantly better than the ^Jn performance loss (in terms of samples 
needed) derived from just first and second order statistics at all the p d values covered in these 
figures. 
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Figure 2-2: Yincoh/Ycoh as function of p d for various p fa . 
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Figure 2-4: p energy /Yincoh as function of p d for various Pf a with n = 100. 
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Figure 2-5: p energy /Yincoh as function of p d for various Pf a with n = 1000. 
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Figure 2-7: n energy /n incoh as a function of p d with p^ a = le — 3 and p = — 5 dS. 



14 



12 



510 



8 



— n /n. 

energy incoh 
1/2 

—n 

energy 



0.75 0.99 

Probability of Detection 



Figure 2-8: n energy /n incoh as a function of p d with p^ a = le — 5 and p = — 5 d5. 



24 



3 Robust Detection 

3.1 Detection when the waveform and the initial phase are known 

3.1.1 Case where contamination occurs over data pairs 

Let 3 t and 3 be two disjoint classes of n-dimensional density functions, and let 3 1 and 3 
represent respectively the hypotheses H 1 and H . Assume that x n is generated by a single 
density function in 3 t U 3 . Furthermore, consider that the two classes are Huber classes 
defined as 

3> = {fixuyd = (1 - £ )fo(xi,yi) + e h(x t ,yi) (Eq. 3.1.1) 

3i = {fi.Xi.yd = (1 - £i)fi, (.Xi,yd + £iKx u yd (Eq. 3.1.2) 



where 



fobuyo = i^ exp {wM + yf)) (**• 3-i-3) 



fu(x t ,y t ) = ^~ ex P \^~{ x i ~ ^ C0S (Pi + °o)) 
— (yi - yfEsinipi + 9 j) J 



(Eq. 3.1.4) 



+ 2N n 



and where s , s r are two given real constants whose values lie in (0,1)- The classes above model 
the cases where a sequence of data is generated by a corresponding nominal process, and 
contaminations may appear independently per data pair (x^y^) with probabilities s for class 
3 and s t for class 3 t . We later consider the case where the contamination may occur 
independently per datum x t or y t . 
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Based on the results derived by Huber, as presented in Detection and Estimation (169-74) [3], 
and extended in Appendix A, we have 



* fey ' )=lo8 teo^)j (Eq - 3 ' 1 - 51 



( l^~ £ 1 N 



z -) x u y t \ Pi(x u yi) < lnc 1; j 

1 - e J 

Y^f-j+Pi&uyd xi,y t : In c u < faOc u y t ) < \r\c oi 3 

1 - £l N 



(1 c \ 
^ -) Xi.yf. Piixi.yd >\nc oi 

1 - e J 



where 



Pt(xt,yt) = — ^2x ; - yJEcosipi + 6 )j 



+ 



2N 

yfEsinipi + ej, r- . , , (Eq " 3 - L7) 



2iV n 



■ (2yj - yfEsinipi + 9 ) J 



Let us define 



k t = XiCos(pi + 9 ) + yisinipi + O ) 



Then 



N l 2N 2N 



(Eq. 3.1.8) 



fo(x u yd = Pifri) = —k t - — = — (2k t - VI) (Eq. 3.1.9) 



The statistics of k t under /i,i(fci),/ (fci) are the same for all £ and therefore the statistics of /q 
under fliQci), fo,i(ki) are the same for all i. Thus, from this point on, we can drop the index i 
from yl, c ti , c oi , /? ; . Let us then define 
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z(/,)= ;/ =K(/,)- 7 =ln( I - 1 -) + - 



(Eq. 3.1.10) 



d 1 = -ln Cl+ - 



(Eq. 3.1.11) 



d = ^lnc 0+ - 



(Eq. 3.1.12) 



Then 



z(fc t ) = fcj 

I d 



k t < d t 
d-i < k t < d 

k t > d 



(Eq. 3.1.13) 



The analysis has been simplified down to the case where a deterministic scalar constant signal 
may be transmitted in additive white Gaussian noise and follows Example 6.3.1 presented in 
Detection and Estimation. The key to this simplification lies in the assumption that 
contamination occurs over data pairs ipci,y{). We include some of the key results and equations 
of the analysis found in Detection and Estimation as future sections will reference them. The 
reader should refer to Detection and Estimation (p. 175-85) for more details [3]. 

We will focus our analysis on the case where s = s t = e < 0.5. 

The robust decision rule is 



<r(/t") = 



1 


k n 


i = l 


'(k t ) > r 


r* 


k n 


n 

:n-i Z- 

i = l 


z(k t ) = r 





k n 


n 

n_i Z 2 


■0c t ) < r 



(Eq. 3.1.14) 



i = l 
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where z(/q) is as in (Eq. 3.1.13) and where A* and r* are such that 



Pr In' 1 V z(/q) > X* f * + r* Pr In' 1 Y z(fc t ) = T 



/o* = a (Eq. 3.1.15) 



The least favorable density functions / * and /j* are 



/o (« = 



(1-e) / ^ , 

:expl— — — I k:k<d 



4^k 



2N r 



(1-8) f -{k-4E) E Vff \ 

exp — — 1- — — ~^-d k:k>d 



(Eq. 3.1.16) 



J2M~ 



2N n 



2N N 



A* 00 = 



:exp — — k:k>d 1 



4l^N~ 



2N n 



(1-e) f-k 2 E VI . 

ex P ( ~^Tt — ^t^ + tt - ^i ) k-.k<d 1 



.J2^r 



2N 2N N 



(Eq. 3.1.17) 



where 



2 _1 VI r > d = di = -d + VI 



(Eq. 3.1.18) 



and d is the unique solution of 



(1-e) 



d IE \ fdVE 

®\ ~^= + I— l + exp 



JV 2JV 



£ \ / d 




= 1 (Eq. 3.1.19) 



where there is the solution to (Eq. 3.1.19), which results in a valid d if 



£ < £ b = 1 - 2 _1 







Ve 

2jJL 



(Eq. 3.1.20) 
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Since the variables {K t } are independent and identically distributed when conditioned on any / 
in 3-i U 3 , the random variables {/?; = z(if;)} are too. The random variable n _1 £f =1 z(i^) is 
then asymptotically Gaussian. Let us denote 

p = 4E/jN~ (Eq. 3.1.21) 

c(e) = d/yfN^ (Eq. 3.1.22) 

Denote m.f* and vj* as the mean and variance of the random variable n _1 £f =1 z{K{) as n gets 
large, conditioned on / *. We find 

m /o . = (1 - £)V^o(<P[c(e)][c(e) + [p - c(s)]exp[pc(s) - 2~ 1 p 2 ]] 

li4fni , r r ^ u (Eq. 3.1.23) 

i?2, = (l - e)N rT 1 [<P[c(e)] [c 2 (e) + (p - c(e)) 2 exp(pc(e) - 2 _1 p 2 ) 

-l] + ^[p-c(£)]+c(£)0[c(£)] + [c(£)-p]0[p-c(£)] 

(Eq. 3.1.24) 
-(l-£)-X"Xo*} 

For asymptotically large values of n, the constant r* in (Eq. 3.1.15) can be taken equal to zero 
and we obtain 

X* = m f5 + v f5 ®-\l-a) (Eq. 3.1.25) 

Denote m^ and vj as the mean and variance of the random variable n _1 £f =1 z(K{) as n gets 
large, conditioned on f t . We find 
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m A = 4^\P ~ 0( c ( £ )) + <K C ( £ ) - P) + (c(e) - p)<X.c(e) - p) 
-c(e)<P(c(e))] 



(Eq. 3.1.26) 



v 2 h = n - 1 N [c 2 (is) + [1 + p 2 - c 2 (e)]0[p - c(e)] 

+ (c 2 (s) - 2pc{e) - l)<P[c{e)} - [-c(e) + 2p]0[c(e)] 
+ [p + c(e)]0|> - c(e)] - iV-im^} 



(Eq. 3.1.27) 



Denote by m 5m and v| m respectively the mean and variance of the random variable 

ft -1 Zf=i z(Ki)> as n 8 ets l ar g e < conditioned on g m (K) , where g m is Gaussian with variance N 
and mean m. Define 



p = m/JNg 



(Eq. 3.1.28) 



We find 



m, 



9m 



= /^{p +[p-H- c(e)] <P(c(e) -p + fi) 

+ [c(e) - p] <P(c(e) - p) + 0(c(e) - p) 

-0(p-p-c(f))} 



(Eq. 3.1.29) 



Z7| m = n-^Jl + p 2 + [c 2 ( £ ) - p 2 - l]<P[c(£) - p] 

+ [(c(e) - p) 2 - p 2 - 1 ] <P[c(e) - p + p] 

+ [c(e) + p]0[c(e) - p] + [c(e) - p - p]0[p - p - c(e)] 



(Eq. 3.1.30) 



For 
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f = (l-e)f 1 + eg n (Eq. 3.1.31) 



we obtain 



mf = (1 - s)m fl + sm gm (Eq. 3.1.32) 

vf = (l- £)v 2 h + ev£ m + n _1 e(l - s)[m fl - m gm ] 2 (Eq. 3.1.33) 

Let us denote P n (f) the power induced by the robust decision rule in (Eq. 3.1.15) at the density 
function / in (Eq. 3.1.31). Due to (Eq. 3.1.25), we conclude: for n -* oo, 



frrif — m f * — v f *<P 1 (1 — a)\ 
Pndf) = <P — " (Eq- 3.1.34) 



3.1.2 Case where contamination occurs per datum 

For this case, where the waveform is completely known, treat each complex pair as an individual 
sample. Define 

U T = (x 1 ,y 1 ,x 2 ,x 3 ,....,x n ,y n ) (Eq. 3.1.35) 

Also define 

hl n 

= (Vlcos(0 o + p(l)) , Vising 3 i 

+ p(l), ... , Vlcos(0 o + p(n)) , Vlsin(0 o + p(n))) 

In other words, h\ n is the known signal. With this new notation, 
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^ = exp — (2u,- ft ,) (Eq.3.1.371 



Our problem can be stated as follows. Let 3 t and 3 be two disjoint classes of n-dimensional 
density functions, and let 3 1 and 3 represent respectively hypotheses H 1 and H . Assume that 
u 2n is generated by a single density function in 3 t U 3 . Furthermore, consider that the two 
classes are Huber classes, defined as 



3 = {/(Mi) = (1 - e )f (u t ) + s h(Ui) (Eq. 3.1.38) 



^i = {/(";) = (1 - £i)/i,i("i) + eihOk) (Eq. 3.1.39) 



where 



foM = ^™p(wS U V) (Eq. 3.1.40) 

flM) = ysf xp ^> {Ui ~ hiY ) (Eq - 3AA1) 



and where s , s t are two given real constants whose values lie in (0,1). The classes above model 
the cases where a sequence of data is generated by a corresponding nominal process, and 
contaminations may appear independently per datum (u ; ) with probabilities s for class 3 and 
s t for class 3 t . 

Based on the results derived by Huber, as presented in Detection and Estimation (169-74) [3], 
and extended in Appendix A, we have 



(f\i(ui)\ h: , 
VoAuOJ 2N 
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, f , , ( fiMd 
Yi{Ui) = log{ KMJ 



(Eq. 3.1.43) 



lnC ^ +ln (bf) 



> ln ^ +ln (l^) 



Ui'.Piiui) <lnc 1;i 
uo In c u < ft(u £ ) < lnc 0|i (Eq. 3.1.44) 
UfPiiui) > In c oX 



where 



Then, if h t > 



or iffy < 



PM) = 2Jf(2ut-hi) 



(Eq. 3.1.45) 



Let us define 



^ ) = V^-|K^t)4 ^ 3 - L «) 



Wn 



/l ; 



1,1 ^ 1,1 2 



(Eq. 3.1.47) 



N n 



hi 



d ,i = T- lnc o,i + v 

hi 2 



(Eq. 3.1.48) 



di,i Ui < d u 

Zi(Ui) = \u t d t < u t < d Qi 

d ,i u t > d 0ii 



(Eq. 3.1.49) 
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Zi(Ui) = \Ui 



di,i u t > d u 

d t > u t > d oi 
d ,i "i ^ d i 



(Eq. 3.1.50) 



We will focus our analysis on the case where s = s t = s < 0.5. 



The robust decision rule takes the form 



S*(u n ) = 



2n 



1 u 2n :2- 1 n- i yh i z i (u i )>X* 

i = l 
2n 

r* u 2n :2- 1 n- i yh i z i (u i ) = A* 

i = l 
2n 

u 2n :2- 1 n- 1 \h i z i (u i )<A* 



(Eq. 3.1.51) 



i = l 



where Zi(u{) are as in (Eq. 3.1.50) and where A* and r* are such that 



!2n \ r n 

2- x n~ x V hiz{ui) > A* /o j + r* Pr 2 -1 n -1 Y ^z(uj) = A* 



f *\ = a (Eq. 3.1.52) 



The least favorable density functions / * ; and f^ t , if h t > are 



/o,i("i) = 



(1-f) 



:exp 



2iV f 



72^ v —o 
(1 - £ ) /-(u £ - h ; ) 2 _ h 2 ^ 



Ui'.Ui < d 0i i 



•V^Vo 



exp 



2AL 



+ ^7T _ 7r d o,i Ui'.Ui >d oi 



2N N 



(Eq. 3.1.53) 



KM) = 



( (l- £ ) / (u ; -/i ; ) 2 \ 
(1 - e) f-uf h 2 hi 



4?M~ 



exp 



2N 2N N 



+ TT d i,i uc.Ui < d u 



(Eq. 3.1.54) 



and if /ij > are 
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/o,i("i) = 



(1-e) 



exp 



2N r 



u i'- u i > "o,i 



(1 - e) /-(uj - hd 2 h 2 hi 

ex P ™ + ^777" ~7T d o,i Ui:Ui<d 0ii 



.f2M~ 



2N n 



2N N 



(Eq. 3.1.55) 



f (1-e) / (u t -fc t ) 2 \ 

ex Pl 777 ) Ui-Ui<d u 



fiM) = 



Jhrff 



2N n 



(1 - s) f-uf h 2 h t 



.J*nN~o 



exp 



2N n 



2N N 



+ TT d U Ui\Ui> d u 



(Eq. 3.1.56) 



where 



di = d 1 = —d + hi 



(Eq. 3.1.57) 



and {d t } are the solutions of 



(1-e) 



. , d t hi \ fdthi hj \ ( d t 



N 2N 



1; hi.hi >0 (Eq. 3.1.58) 



(1-e) 



, , d t hi \ fdthi hj 

<P\^= t= 1 + exp' 



N 2N 







-di 



= 1; hi.hi <0 (Eq. 3.1.59) 



From (Eq. 3.1.51), we observe that the robust receiver performs a truncation per datum in a 
similar fashion as that of the robust receiver found in section 3.1.1, except that the function 
which performs truncation changes here for each sample, according to h t . Also, (Eq. 3.1.49) and 
(Eq. 3.1.50) require that \d t \ < |d |. Due to this requirement, we could have the case where 
some data samples have small enough \h\ so that no valid solution to (Eq. 3.1.58) or (Eq. 3.1.59) 
exists. Those data samples where no solution exists represent a case where 3 lt and 3 ,i 
overlap. For our overall detection, such samples do not help our decision rule, and in the 
analysis that follows we will assign Zi(u.i) = to all samples where no valid solution exist. 
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The variables {f/J are independent when conditioned on any / in 3 1 U 3 . Let us denote a 
subset of {f/J as {U iL •■ [/; where L <h t < L + dh] where dh is arbitrarily small. The members 
in subset {U iL } are independent and identically distributed when conditioned on any / in 
3 t U 3 , and so are the random variables {Y iL = z L (Ui :L )}. The random variable 
n I 1 £"=1 ^i z i(^u)' wnere n L is the number of elements inside the subset, is then 
asymptotically Gaussian. Finally, 2 _1 n _1 J\ =1 h t z (U iL ) is a linear combination of 
asymptotically Gaussian random variables; it is therefore asymptotically Gaussian. 

m fS = E{2- X n~ x ^z^l/o' ) = ^(j h iZ (U,h)o)(h)dh\f*} (Eq. 3.1.60) 

where a)(h)dh is the relative frequency of {f/J such that h < h t < h + dh. For a specified h\ n 
the integral in (Eq. 3.1.60) will reduce to a sum. Let us assume that, as n gets asymptotically 

large, (6*)d9* = —,-n<9*<n; where 6* = 9 + p(i) for x t samples and 6* = - - 6 - 

p(i) for y t samples. Our model will be asymptotically correct, if the signal is such that, as n gets 
large, the relative occurrence of a sample falling into any phase band of width d6* is equally 
probable. Within the latter model, (Eq. 3.1.60) can be written as, 

m fS =^ E{z{U,B*)f: fi ,}dB* 

(Eq. 3.1.61) 



= ^j h i E{z(U,h = Vlcos(0*) \f; }h )}d6* 



Let us denote 



p(h) = \h\/jN~ (Eq. 3.1.62) 
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c(s, k) = d{h)/4N~ (Eq. 3.1.63) 

With this notation, and based on (Eq. 3.1.23), we find 

E{z{U,h)\fo,h) 

= (1 - e)jN^{0[c(e,h)][c(e,h) + [p(h) - c(s,h)]exp[p(h)c(s,h) 

-2-V00]] +0[c(£,/i)] (Eq. 3.1.64) 

- (p[c(e,K) - pQi)]}; s b Qi)>s 

E{z{U,h)\r oh } = Q; e b (h)<e 



Define h b {e) = mm{\h\: e b (h) > s}. Define 6 b {e) = cos 1 (-j=-) ■ Due to symmetry, (Eq. 
3.1.61) can be then simplified to the following form, 



2 r n/2 

-.=- 4Ecos(6*)E{z{U,4Ecos(e*)\fZ h )}de* (Eq. 3.1.65) 



Since we must find c(s, h) numerically, we must also evaluate (Eq. 3.1.65) numerically. In a 
similar fashion, and based on (Eq. 3.1.24), we find 

2 f n/2 
vl = (In)' 1 - E 2 cos 2 (9*) Var{z(U,VE cos(6*) \f* h )}d9* (Eq. 3.1.66) 

nJ e b (8) 

Var{z(U,h\fo,n)} = 

(1 - s)N J 0[c(s, h)] \c 2 (e, h) + (p- c(s, hjf exp(p(/i)c(>, K) - 

2—lp2(h)—l+<I>p(h)—c£,h+c£,h(fic£,h+c£,h—p(h)(pp(h)—c£,h—l—£— 
lNo-lE2zU,hlfoM 



37 



For asymptotically large values of n, the constant r* in (Eq. 3.1.52) can be set at zero, resulting 



in 



A* = m f * + v f * 1 (1 - a) (Eq. 3.1.68) 

Asymptotically, 2 _1 n _1 £. =1 h t z (JJ{) is Gaussian, when conditioned on 
/ = (1 — e)f lt + sg m ; where g m} i(ui) is Gaussian with mean mcos(6*) and variance N . 
Define //(/i) = mh/^N E) = mcos{9*) / 'JW . In (Eq. 3.2.26), (Eq. 3.2.27), (Eq. 3.2.29) and 
(Eq. 3.2.30) we can replace p with p(h), c(e) with c(s, h), // with /i(7i) and n with 2n and 
integrate in a similar fashion, as in (Eq. 3.1.65) and (Eq. 3.1.66), to obtain the mean and variance 
(as n -» oo) of 2 _1 n _1 S i=1 h t z ([/;) ; when conditioned on / = (1 — s)f ti + sg mi i. 

m A =- 4Ecos{9*)E{z{U l 4Ecos{9*)\f xh )}dB* (Eq. 3.1.69) 

vj = (2n) _1 - E 2 cos 2 (0*)Kar{z(f/,VIcos(0*)|/ lh ))d0* (Eq. 3.1.70) 

n J 



2 r0 b 00 



m s m 



2 r u bitj 
= - VIcos(0*)£{z(£/,VIcos(0*)|# m )}d0* (Eq. 3.1.71) 

77 Jo 



2 rSfiO) 
i^ m = (2n)- 1 - £ ,2 cos 2 (6'*)yar{z([/,VIcos(6'*)|^ m )}d6'* (Eq. 3.1.72) 



E{z(>, fc|/ 1#ft )} = ViV7[p(7l) - 0(c(£, /l)) + 0(C(£, /l) - p(/l)) 

+ (c( £ , /i) - p(/i)) ^(c( £ , h) - p{K)) - c{e, h) 0{c{s, h))] (Eq ' 3 ' 1 - 73) 
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Var{z(U,h\f lih )} 

= N [c 2 (e, h) + [1 + p 2 (/i) - c 2 (£, /i)] ${p{h) - c(s, h)] 

+ (c 2 (s,h) - 2p(K)c(e,K) - l)0[c(s,h)] 
- [-(c(e,h) + 2p(h)]cp[c(e,h)] 
+ \p(h) + c(s, K)]4>[p(K) - c(s, h)] 

-N^E 2 {z(U,h)\h,n}} 

E{z{U,h\g m )} = 4N^{ l i(h) 

+ [p(K) - nQi) - c(s, h)] 0(c(e, h) - p(K) + pi(h) ) 

+ [c(s, h) - fi(h) ] 0(c(e, h) - fi(h) ) 

+ 0(c(£, h) - n(h) ) - <p(p(h) - n(h) - c(s, h))} 



For 



(Eq. 3.1.74) 



(Eq. 3.1.75) 



Var{z(U, V£cos(0*) \g m )} 

= JV {l+// 2 (7i) 

+ [c 2 (s, h) - p. 2 (h) - 1] 0[c(s, h) - n(h) ] 

+ [(c(s,h) - p(h)) 2 - ii 2 {h) - 1 ]0[c(s,h) - p(K) (Eq 3 A ?6) 

+ p.(h) ] + [c(s, h) + pl(K) ](p[c(s, h) - nQi) ] 

+ [c(e, h) — p(h) — n(h) ]<fi[p(h) — fi(h) — c{e, h)] 

-N- 1 E 2 {z{U,K)\g m }} 



f = (1 - s)f u + eg m (Eq. 3.1.77) 
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we have 

m f = (1 - e)m fl + em gm (Eq. 3.1.78) 

v} = (l- e)v\ + £vj m + n^etX - e)[m fl - m g J 2 (Eq. 3.1.79) 

Let us denote by P n (f), the power induced by the robust decision rule in (Eq. 3.1.52), at the 
density function / in (Eq. 3.1.77). Due to (Eq. 3.1.68), we conclude: for n -* oo, 



frrif — rrif* — v f *<P 1 (1 — a)\ 

p n iO - <H— — *—^ — - — -) ( £ q- 3- 1 - 80 ) 



3.1.3 Comparison of the robust cases with the optimal at the nominal model test 

In section 2.2, we examined the Neyman-Pearson detection rule at the nominal model. The 
latter rule does not provide protection against data contamination represented by extreme 
outliers. In section 3.1.1, we examined the robust detection scheme in the case where 
contamination occurs per data pair(x £ ,yj). In section 3.1.2, we examined the robust detection 
scheme in the case where contamination occurs independently per datum. In this section, we 
will compare the asymptotic performance of all three detection schemes under all three 
contamination models (no contamination, contamination occurs per data pair, and 

X\Y 

contamination occurs per datum). We will use the notation P n to denote the power of the 
detection scheme X under the contamination model Y, where X and Y can be A, B, or C to 
respectively represent the cases of no contamination, contamination occurring per data pair, or 

D I /" 

contamination occurring per single datum. For example, P n represents the power of the 
robust detection rule derived assuming contamination occurring per data pair (B) under the 
model where contamination actually occurs per single datum (C). More specifically, for X = B, 
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we will consider the power of the detection rule under/, as defined in (Eq. 3.1.31); and for = C, 
we will consider the power of the detection rule under /, as defined in (Eq. 3.1.77). 

Comparison under the model of no contamination 



Denote p = / — . From (Eq. 2.2.8), we have 

V N o 



P^ A = 0(pVi+ 0~ 1 {a)) (Eq. 3.1.81) 



Based on (Eq. 3.1.34), we have 



(m f B — m f ,B — v f ,B0 1 (1 — a) N 

■)B\A _ A | /l To Jo 



P^ A = 0[ -± Jo (Eq. 3.1.82) 



where m f *B, v**b, m f B, and v f B are defined in (Eq. 3.1.23), (Eq. 3.1.24), (Eq. 3.1.26), and (Eq. 
3.1.27). 



Based on (Eq. 3.1.80), we have 



r\A ( m f c ~ m r c ~ v r c@ 1 (1 - a)\ 
P^ A = 0\-^ * ± (Eq. 3.1.83) 

V V A C J 



where m f «c, v f ,c, m f c, and v f c are defined in (Eq. 3.1.65), (Eq. 3.1.66), (Eq. 3.1.69), and (Eq. 

Jo Jo Jl Jl 

3.1.70). Figure 3-1 shows the power curves for parameters: E = 0.25, N = 1, s = 0.1, a = 
0.05. Figure 3-2 shows the power curves for parameters: E = 0.25, N = 1, s = 0.01, a = 0.05. 

A\A B\A C\A 

It can easily in both Figure 3-1 and Figure 3-2 that P n > P n > P n for all n values plotted. In 
other words, Figure 3-1 and Figure 3-2 show that the optimal detection scheme out performs 
the robust detection scheme if there are no outliers. 
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Figure 3-1: Power curves for E = 0.25, N = 1, e = 0.1, a = 0.05. 
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Figure 3-2: Power curves for E = 0.25, N = l,e = 0.01, a = 0.05. 
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Comparison under the model of contamination per data pair 
From Detection and Estimation (p. 184) [3], we obtain 



Pn = M , (Eq. 3.1.84) 



V Vl + e(l-£)0>-/x) 2 
From (Eq. 3.1.34) we have 



„.„ (m fB -m f , B - v f ,B0 1 (l-a)\ 
P„ B|B = <pM /o v f ° -J (Eq. 3.1.85) 



where m f *B, v^b, m f B, and v f B are defined in (Eq. 3.1.23),(Eq. 3.1.24),(Eq. 3.1.32), and (Eq. 
3.1.33). To derive P n , we consider density functions defined by 

f(x t ) = (1 - e)f u 0c t ) + £g m ,x,i( x i) (Eq. 3.1.86) 

f{yd = (1 - z{i))f u {yd + z(i)gm, y ,i(yi) (Eq. 3.1.87) 

where 

(1; if Xi was a realization of q m r ,(x,) 

z(i)= ' ./ ,. ,. f?T\ (Eq. 3.1.88) 

( 0; if x t was a realization of fniXi) 

These result in 

fixi.yd = (1 - e)fu(.xi,yi) + Eg mA (Xi,yi) (Eq. 3.1.89) 

Furthermore, define g m ,x,;( x i) to be Gaussian with mean mcos{6 + p(0), and variance iV , 
and g m , y ,i(x{) to be Gaussian with mean msin(6 + p(i)) and variance N . We obtain 
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fik t ) = (1 - £)A(fei) + eg m (ki) (Eq. 3.1.90) 

where /i(/ti) and g m (ki) are the same densities as in Section 3.1.1. In other words, the 
densities defined by (Eq. 3.1.86) and (Eq. 3.1.87) result in the same density model assumed 

under contamination model B. We therefore can find P n as n -» oo by examining the statistics 
of 2 _1 n _1 Y^2i h t z c ({/() conditioned on (Eq. 3.1.86), and (Eq. 3.1.87) , where z c (u t ) is 
defined in (Eq. 3.1.49) and (Eq. 3.1.50). The density defined by (Eq. 3.1.86) and (Eq. 3.1.87) is 
similar to the density model assumed under contamination model C, except that the outliers 
always occur on consecutive values of u t . The statistics of 2 _1 n _1 £. =1 hiz c (JJ{) as n -* oo will 
be the same whether outliers always occur consecutively or not. Therefore, we conclude that 
P n c|B = p^ c . From (Eq. 3.1.80), we have 

n R (m fC -m r c-v r c@~ 1 (l-a)\ 
P^ B = 0i-L h b. _ 1 (Eq. 3.1.91) 

where m f ,c, v f ,c, m f c, and v f c are defined in (Eq. 3.1.65),(Eq. 3.1.66),(Eq. 3.1.78) and (Eq. 

JO JO J I 

3.1.79) respectively. Figure 3-3 shows the power curves for parameters: E = 0.25, N = 1, s = 
0.1, [i = —5. Figure 3-4 shows the power curves for parameters: E = 0.25, N = 1, s = 

0.01, n = -5. It can easily be in both Figure 3-3 and Figure 3-4 that P„ B|B > P* |B and P„ B|B > 

P n for all n values plotted. In other words, Figure 3-3 and Figure 3-4 show that the robust 
detection scheme based on the appropriate outlier model outperforms the two other detection 
schemes examined. 
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Figure 3-3: Power curves for E = 0.25, N = 1, e = 0.1, a — 0.05, \i = —5. 
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Figure 3-4: Power curves for E = 0.25, N = l,e = 0.01, a = 0.05, \i = -5. 
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Comparison under the model of contamination per datum 

To find P n , we need to examine the statistics of n -1 £f =1 K t where k t = x t cos(0 o + p(0) + 
y;sin (0 o + p(i)) under 

fiixuyd = (i - £) 2 fi,iixi,yi) 
+ s(l - £)[h,i{xi)g mA {yi) + g m ,i{xi)fi,i(yi)] , £ 3 ± g2 j 

+ £ 2 g m , i (x i )g m}i (y i ) 

Define 

qiiki): Probability density of k t |A,iOi)#m,i(yi) (Eq. 3.1.93) 

niki): Probability density of k { |# m ,iOi)A,i(yi) (Eq. 3.1.94) 

Si(ki): Probability density of k t |# m ,iOi)#m,i(yi) (Eq. 3.1.95) 

Define mf,vf as the mean and standard deviation of n _1 £f =1 Ki as n -* oo , conditioned on 
density/ where j can be q,r,s , denoting respectively the densities in (Eq. 3.1.93), (Eq. 3.1.94), 
and (Eq. 3.1.95). We note that n~ 1 'Zf =1 K i , conditioned on q, r or s, is a linear combination of 
Gaussian variables and is thus Gaussian. From Section 3.1.2, we note that g m ,i(u;) is Gaussian 
with mean mcos(8 + p(0) ; for X; data values and with mean msin(8 + p(0) ; for yi data 
values, while, both x t and y t data values have common variance N . Denote \i = max(/i(7i)), 
where fi(h) is defined in Section 3.1.2. We then find, 



i = l > 



<~ E \ n ~ ^'Ml^j (Eq. 3.1.96) 



1 f zn 
= ^ J E(xi\f u ) cos(0) + E ( yi \g m (y 0)sm (0) dO 

= 2- 1 V£ r + 2 _ V 



In a similar fashion, we find, 
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Vq = Vr ~ v s = V n_1 ^o 



(Eq. 3.1.97) 



m 



* = 2-^ + 2- 1 



/" 



(Eq. 3.1.98) 



mj =}i 



(Eq. 3.1.99) 



Define m^|c, and v^|c as the mean and variance of n x £f=i ^Q ; as n -* oo , conditioned on the 
density in (Eq. 3.1.92). We then find, 



m 



,A\C = (1-£)VI + 



S/I 



(Eq. 3.1.100) 



v 2 fA \c = n 1 






(Eq. 3.1.101) 



a\c _ (m fA \c-Jn- 1 N o 1 (l-ay 






V^|C 



(Eq. 3.1.102) 



Denote by mf ,vf as the mean and standard deviation of the random variable n 1 T,f=i z(Kj) ; 
as n gets large, conditioned on density j ;where j can be q, r, s, referring respectively to the 
densities in (Eq. 3.1.93), (Eq. 3.1.94) and (Eq. 3.1.95). We note that k t is Gaussian when 
conditioned on q, r or s, with variance N . We will define [ij(6*) to be the mean of k(6*) 
divided byJiV^, when conditioned on density y' ; where/ can be q, r or s. We find 
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1 f 2n 
mf = — £{z(tf(0 *)[/')} dd* (Eq. 3.1.103) 

2n J n 



1 r27z: 

(vf) 2 = (n)- 1 — 7ar{z(ff(0*)|y)}d0* (Eq. 3.1.104) 

Z7T J Q 



E{z(K(e*)\))} = jN^{nj{6*) 

+ [p - ^-(0*) - c(e)]<p(c(£) - p + ax 7 -(0*) ) 

+ 0(c(e) - //,(0*) ) - (p - ax 7 -(0*) - c(e))} 



(Eq. 3.1.105) 



Var{z{K{9*)\))} 

= JV o {l + M 2 (0*) 

+ [c 2 (s)-fij(9*) - l]®[c{e) - nj(e*j\ 

+ [(cO) -p) 2 -/if(0*) - l]<p[c(e) -p + fijW)] 

+ [c(e)+n j (e*)]<l>[c(e)-nj(e t )] 

+ [c(e) - p - M;(0*)]0[p - M;(0*) - c(e)] 

-iV-M^z^C^)!/)}] 2 } 



(Eq. 3.1.106) 



where 



[i q (9*) = JV o 1/2 [VIcos 2 (0*) + msin 2 (0*)] (Eq. 3.1.107) 



/i r (0*) = N o 1/2 [VIsin 2 (0*) + 7ncos 2 (0*)] (Eq. 3.1.108) 



fi s (9*) = N 1X m[cos 2 (0*) + sin 2 (0*)] = \i (Eq. 3.1.109) 
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and where (Eq. 3.1.105) and (Eq. 3.1.106) are obtained from (Eq. 3.1.29) and (Eq. 3.1.30). For 
mf and (vf ) 2 , the dependence of 8* disappears, and mf and (yf ) 2 are identical to m 5m and 
Vg m , which are found in (Eq. 3.1.29) and (Eq. 3.1.30) . Define m f B\c and v*b\c as the mean and 
variance of n~ 1 'Zf=iz(K i ) ;as n -* oo , conditioned on the density in (Eq. 3.1.92). We then find 



m f B\c = (1 - s) 2 m f B + e(l - s)m^ + e(l - s)m.r + s 2 m^ (Eq. 3.1.110) 



v)b\c = (1 - e) 2 v 2 f B + e(l - £)[v%} 2 + 8(1 - £ )K B ] 2 + s 2 [v?] 2 + 
_1 (1 - e) 2 £(1 - e) ( m f B - mf] + ^m f B - m B ] J 

+ s 2 [m f B - mf ] } + (Eq. 3.1.111) 

n~ 1 s 2 ie(l - s) ([mf - m?] 2 + [mf - mf] 2 )) 
+ n _1 £ 2 (l - E~) 2 [m? - m%]' 



n 



l2 



and 



„.. (m fB \c - m f , B - v f , B 1 (1 - a)\ 
P* lc = <p[^ f -° 10. (Eq. 3.1.112) 

V v f Blc J 



where m f *B, and v**b are defined in (Eq. 3.1.23) and (Eq. 3.1.24). Finally, from (Eq. 3.1.80) we 
have 



r\r ( m f c ~ m r c ~ v r c ° 1 ( 1 ~ a )\ 
P n c|c = <Z>M * in. (Eq. 3.1.113) 

V v f c J 



where m f ,c, v f ,c, m f c, and v f c are defined in (Eq. 3.1.65),(Eq. 3.1.66),(Eq. 3.1.78) and (Eq. 

J0 JO J J 

3.1.79), respectively. Figure 3-5 shows the power curves for parameters: E = 0.25, N = l,s 
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0.1, n = —5. Figure 3-6 shows the power curves for parameters: E = 0.25, N = 1, s = 

C\C A\C 

0.01, n = —5. It can easily be seen in both Figure 3-5 and Figure 3-6 that P n > P n and 

P-n > Pn f° r a " n va l ues plotted. In other words, Figure 3-5 and Figure 3-6 show that the 
robust detection scheme based on the appropriate outlier model outperforms the two other 
detection schemes examined. 
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Figure 3-5: Power curves for E = 0.25, N = 1, e = 0.1, a = 0.05, \i = —5. 
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3.2 Detection when the waveform is unknown (energy detection) 

Let S t and S be two disjoint classes of n-dimensional density functions, and let 3^ and 3 
represent respectively hypotheses H 1 and H . Assume that x n ,y n are generated by a single 
density function in 3 t U 3 . Furthermore, consider that the two classes are Huber classes 
defined as 



3 = if(xi,yi) = (1 - £ )fo(.Xi,yi) + Eohipci.yi) 



(Eq. 3.2.1) 



where 



3i = {f&i.yi) = (1 - £i)fi,o( x i>y^ + £ iKxi,yi) (Eq. 3.2.2) 



fo<*.yd = i^exp (-^-(xf + yf)) (Eq. 3.2.3) 
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fi(xt,y t ) = 



2nN, 



expl 



2N, 



1 ■> 7 E 

(xf+yf)- 




(Eq. 3.2.4) 



and where s , s t are two given real constants whose values lie in (0,1)- The classes above model 
the cases where a sequence of data is generated by a corresponding nominal process, and 
contaminations may appear independently per data pair (xi,y{) with probabilities s ; for class 
3 and e 1 ; for class 3 t . 



Y(xi,yd = ln 



fiJXj.yd 
fo(xi,yi) 



I 



2AL 



+ ln 



'JxfE + y?E\ 



\ 



N n 



7 



(Eq. 3.2.5) 



define u t = xf + yf. Then 



Y*(u{) = log 



/o(u £ ) 



(Eq. 3.2.6) 



In c x + In 



\l - sj 






ta H + « , * ) 



lnc » + ln (r^l) 



Ui-.piiii) <lnd 
u ; : In c t < p(ui) < In c 



Ui-.piui) >lnc 



(Eq. 3.2.7) 



where 



E I (JujE s 



(Eq. 3.2.8) 



Let us define 



Then 



Finally, define 



Then 



The robust detection rule can be written as 
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Zi = lo 



yfuiE 



N n 



(Eq. 3.2.9) 



d ^ = c ^ exp {ir) 



(Eq. 3.2.10) 



do = c ° exp {ir) 



(Eq. 3.2.11) 






(Eq. 3.2.12) 



ln " + ta (i^f) 



/l-£i\ E 

lnc » + ln (l^t) 



Zi'.Zi < d 1 



z i :d 1 <z i <d (Eq. 3.2.13) 



Zi-.Zi > d 



a( Zi ) = Y *(zd- In (yZJ-) + 



ej 2N 



(Eq. 3.2.14) 



!lnd 1 
lnzj 
lneL 



zv-Zi < d 1 

z i :d 1 <z t <d 

zc.Zi > d 



(Eq. 3.2.15) 



T*(u n ) = n- 1 ]T \a( Ui ) + In (yzj) ~ 

i = l 



s J 2N 



(Eq. 3.2.16) 
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S*(u n ) = 



1 


u n 


n 




t(iii) > A* 


r* 


u n 


:n 


n 


x(ui) = A* 





u n 


n~ 


n 

'I' 


t(ui) < A* 



(Eq. 3.2.17) 



i = l 



where A* and r* are such that 



Pr In' 1 V a(ui) > A* /„* j + r* Pr n _1 V a{u{) = A* 



f *\ = a (Eq. 3.2.18) 



The least favorable density functions / * and /j* are 



/o 00 = 



(1 ~ g o) 
2N n 



-exp 



V 2JV / 



u:/ n 



N n 



< d 



(1 — £ ) / — u \ (4uE 



pyn 

L 2d o N ^,, 



SK 



N n 



u:L 



N n 



>d 



(Eq. 3.2.19) 



fi(u) = 



f (l-£i) /-U 

(1 - gjdi 




v 2N 



f—u — 1\ 



(Eq. 3.2.20) 



and the constants d and d t are such that 



(l-£o)[Pr \I 



N, ' <d ° 



fo 



+ 



ex P\2N:) i HUE 



121 Pr/, 



N n 



>d 



A ] = i 



(Eq. 3.2.21) 
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(l- £l )[Pr \l — - \>d 1 



N n 



k 



+diexp (-zl:) 



E \ ( HUE , 

Prk(^-)<d, 



(Eq. 3.2.22) 



fo ] = 1 



Define / 1 (x) , such that / 1 (x) = y ; if and only if 7 (y) = x. Define new constants as below, 



N 2 
ko = -^(/o -1 (d )) 2 



(Eq. 3.2.23) 



N 2 



(Eq. 3.2.24) 



With this notation, (Eq. 3.2.15) can be written 



In L 



4^E 



N n 



a(Ui) = < 



In L 



°\ N n 



In I 



. (4ke_ 

°\ N 



Ui-.Ui < k x 



u i :k 1 <u i <k (Eq. 3.2.25) 



Ui'.Ui > k 



Then, (Eq. 3.2.21) and (Eq. 3.2.22) can be written 



exp (2F") 
(1 - e )[Pr{U < k \f } + — §- Pr{U > k \f t } ] = 1 (E 3226) 



(1 - £l )[Pr{{/ > fcJA} + I (^)exp (-^-) Pr{[7 < /q|/ }] = 1 (Eq. 3.2.27) 



or equivalently as; if k , k t > 0: 
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f_E_\ 

(It \ exp I 2 iw I 
-^r) + / ° n Qi(Ve7nT,VWNT)] = 1 , 
" ' STe\ (Eq. 3.2.28) 



! °l '% 



(l-ei)[Qi(VE7N7.V^7N^) 



+, °(Ji) exp (-2^)^- ex K-2^>] =l 



(Eq. 3.2.29) 



We note that if /q < 0, then (Eq. 3.2.27) becomes 1 — £ x = 1; therefore, this will only occur if 
s t = 0. Also we note that we must have k t < k as dictated by (Eq. 3.2.25). 

We will focus our analysis on the case where s = s t = e. 

We will denote the solutions of k , /q in (Eq. 3.2.28), (Eq. 3.2.29), as k (E, N , s) and 
k-iiE.No.s). k (E,N ,s) and k^E.No.e) are continuous since all functions in (Eq. 3.2.28) and 
(Eq. 3.2.29) are continuous over all variables. 

We will define the break-down point s b (E,N ) as the s closest to zero such that k (E,N ,e) < 
k t (E, N , e). We can also denote by k b the k value at which the breakdown point occurs; in 
other words the value when k = k t occurs. From (Eq. 3.2.28) and (Eq. 3.2.29) with k = k t = 
k b and s = s t = s, we conclude that k b must satisfy 



k h E 



'° [jw HKzn:) (Eq - 3 - 230) 



"O / ^"O 



and 
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£b = 



Qi(Ve7n^Vw*u - exp (-^-) 



(Eq. 3.2.31) 



We note that since 



Qi(Ve7n^VWnT) - ex p(-^) 



< 1 



(Eq. 3.2.32) 



then 



£ b < 0.5 



(Eq. 3.2.33) 



There is no known analytic expression for k (E, N , e), k t (E, N , e), k b (E, N ) or s b {E, N ); 
therefore we will use numerical methods to analyze these equations. 
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Figure 3-7: Threshold Values when E < N (JV = 1). 
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Figure 3-8: Threshold Values when E — N . 




Figure 3-9: Threshold Values when E > N (N = 1). 
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30 



Figure 3-10: Threshold Value at the breakdown point. 



CO 




Figure 3-11: Breakdown Point (iV = 1). 



Since the variables {f/J are independent and identically distributed, when conditioned on any / 
in 3 1 U 3 , the random variables {/?; = «(£/;)} are too, and the random variable n _1 £f=i a(JJ{) 
is then asymptotically Gaussian. From (Eq. 3.2.21), (Eq. 3.2.22) and (Eq. 3.2.25) we find 
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1L 

m fS = Ein-^aWM* ) = E{a(U)\f *} (Eq. 3.2.34) 

i = l 



m fS 



-V^^ifw-^-w) 



du 



c 



+ In [In 



VuE\ \ (1 - e) 



N n / 2N n 



■exp 



V 2NJ 



du 



(Eq. 3.2.35) 



i<°m /:+(fw-uMf>» 



+ 



f . = (l- £ )(l-exp(-^-))l„^ 
f ln ('° 



4^E 



4uE\ \ (1 - s) 



No J 2N 



V IN J 



N n 



du 



(Eq. 3.2.36) 



+(!-£) In /, 



m^mf^MMl 



The integral in (Eq. 3.2.36) cannot be written in terms of a well-known function. In the spirit of 
the Marcum Q function we define the following functions 

R t (a,b,c) = I x/n(/ (ax))exp I — — — \dx (Eq. 3.2.37) 



R 2 (a,b,c) = I x[ln(l (ax))) expf-— —jdx (Eq. 3.2.38) 



Then (Eq. 3.2.36) can be written as, 



m/o . = a- £ ,(l-exp(-^)),n(, (^ 



+ (l- c )exp|^WJ* £|£ 



' O A J, A J *0 



+ 



— '»m hffl o 1. 1, 
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(Eq. 3.2.39) 



and 



v£ = ^{n" 1 Y a(t/)|/ * } = n _1 7ar{a(l/)|/ *} (Eq. 3.2.40) 



i = l 



■»^-^( i -^(-ft))W'-( 2 5 

+n - 1(1 _ £)exp (_l_) R2 (JI,JI,^ 



+n _1 (l 



e) In / 



-n 1 ml* 

To 



'yfk^E 



N n 



^mf^HM^ 



(Eq. 3.2.41) 



For asymptotically large values of n, the constant r* in (Eq. 3.2.18) can be set at zero, resulting 



in, 



61 



A* = m fS + v f * 1 (1 - a). (Eq. 3.2.42) 



We will examine the asymptotic performance of our detector under / = (1 — s)f t + sg m ; 
where 



1 / u E \ (yfuE\ 

AW = ^exp (- — - — ) k (— ) (Eq. 3.2.43) 



and 



1 ( u Y \ i\W\ 

3m (u)=^-e X p(- — -—),„(-) (Eq . 3.2.44, 



and where y is a parameter of g m (u), which may be viewed as the energy of the outliers. We 
note that /i(u) is the same density as that given by (Eq. 3.2.4), written here in terms of u 
instead of (x,y). Also, g m (u) results from assuming a Gaussian model outlier under both x and 
y; with means m x and m y , such that y = ml + rriy and variance summing to 2N gm . Let m^, 
vj , nig m , and Vg m denote the mean and variance of n -1 £f =1 a(U{) conditioned on f t or g m , 
respectively. We define the following useful functions 

f b ( x 2 b 2 \ 
R 3 (a,b,c,d) = I x/n(/ (ax))exp I — — — \l (bx)dx (Eq. 3.2.45) 

f d \ 2 ( x 2 b 2 \ 
R 4 (a,b,c,d) = I x f /n(/ (ax))J exp I — — — \l (bx)dx (Eq. 3.2.46) 

We obtain 
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m A = ln 7 ° 



4k^E 



N n 



i-Q 



E_ fci 



+ ln|/„|'^P||<? 



°l /v, ;i vi i jjv n ' U° 



O a O 



(Eq. 3.2.47) 
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In /„ 
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N n 



l - Q ' [ \hk 



"O a O 



+ n 



In /„ 



Vm 7 



N n 



Qi 



\No\No 



(Eq. 3.2.48) 



+ *" 1 *4| l#. l£. &, &l-« -H 
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»m \ »m 



+ In | /„ |'^Z , | (? 



Y k o 



°\~kT 7 PM JvT' ^ 



'm A 'm 



(Eq. 3.2.49) 



EN, 



+ i?. 



7 fci 



N * ' '\'K' ,iV 



v L = n ~ 1 
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yfhE 
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dm \ 9r, 



n ~ ±m L 



(Eq. 3.2.50) 
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/ = (1 - £)A + sg r . 



(Eq. 3.2.51) 



we have 



m.f = (1 — s)m.f x + sm 



9m 



(Eq. 3.2.52) 



vf = (1 - e)v£ + £v| m + n 1 e{l - s)[m fl 



m a 1 



2 (Eq. 3.2.53) 



Let us denote by P n (f>fo) the power induced by the robust decision rule in (Eq. 3.2.18) at the 
density function / in (Eq. 3.2.51). Due to (Eq. 3.2.42), we conclude; for n -* oo, 



Pn(f,fo)=® 



m.f — m.f* — Vf* 1 (1 — a) 



v f 



(Eq. 3.2.54) 



Given a false alarm rate a, let us consider the optimal Neyman-Pearson test at the density 
functions f x and f . By (Eq. 2.3.10) we have 
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P£(fi,fo) = Qnl \^-,^H-\l-a)\ (Eq. 3.2.55) 



where 



H n (x)= j dr 2ni n-l)fi r ~ lexp (-^) (Eq. 3.2.56) 

and H^ipc) is the inverse function of H n (x). H n (x) is the cumulative distribution function of 
the gamma random variable with parameters 2 and n. Denote by m(/) and v 2 (f) the mean 
and variance of U under the density function / = (1 — s)^ + eg m . Then, 

m(f) = (1 - s)E(u\f 1 ) + sE(u\g m ) 
= (1 - £)(£■ + 2N ) + e(y + 2N q J ( Ec "- 3 - 2 - 57 ) 



v 2 {f) = 4(1 - sXEN + JV 2 ) + 4e(yN gm + N*J 
+ s(l-s)(E + 2N - Y -2N gm ) 2 



(Eq. 3.2.58) 



Let us denote by Pn(f>fo) the power induced by the optimal Neyman person rule at/, for 
n -* oo. Directly from (Eq. 2.3.8), (Eq. 3.2.57), and (Eq. 3.2.58), we obtain 



Pfd.fo) = <£(^^ ^jy 1 J —\ (Eq- 3.2.59) 
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Figure 3-12: Probability of detection with E = 1, N = 1, a = 10~ 5 , s = 0.001, y = 0.1, and 



N 9m = o- 1 - 
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Figure 3-13: Probability of detection with E = 1, N = 1, a = 10~ 5 , £ = 0.01, y = 0.1, and 



N am = o.i. 
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In Figure 3-12, we observe that P n (f,fo) > Pn(f>fo) 'J° r a " n examined. However, in Figure 
3-13, we observe that P n (f,fo) < Pn(f>fo) '> f° r a " n examined. The only difference between 
Figure 3-12 and Figure 3-13 is that in Figure 3-12, s = 0.001 while in Figure 3-13 s = 0.01. The 
test in P n (f,fo) seta threshold to guarantee that the probability of false alarm was less than or 
equal to a at / *, and subsequently guaranteed that the probability of false alarm at any /£ J 
was less than or equal to a. In contrast, Pn(f,f ) se t a threshold to guarantee that the 
probability of false alarm was less than or equal to a at f only. Let /? denote the smallest false 
alarm satisfied by the test used to produce P n (f,fo) at/ . The robust detection guarantees that 
fi < a. If fi « a, then comparing P n (f,fo) and Pn(f,f ) ma Y be misleading, since each satisfies 
different false alarm upper bounds at f . Let us examine P n (f,f ), the power of the robust 
decision rule; when the threshold is selected to satisfy the false alarm constraint only at f . We 
then obtain, 



fm.f — m.f — v f 1 (1 — a)\ 
P n (J.fo) = ^\— f - 2 —^ -) (Eq. 3.2.60) 



where 
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(Eq. 3.2.61) 



( E \ I \~E ffci ffco \ 

+n ~ exp l2Jv>(JvJjvJ«;J 




(Eq. 3.2.62) 



+n' 



■'("-('"(^-(-It)-^ 



Figure 3-14 shows that P n (f,fo) > Pn(f>fo) for all n examined, using the same parameters as 
in Figure 3-13. At f t , the optimal test at the nominal model is expected to outperform the 
robust detection rule. The power loss due to the use of the robust detection rule is controlled by 
e, and so is the density used to set the threshold of the test; in satisfaction of the false alarm 
constraint. Figure 3-15 shows P n (fi,fo)> Pn(fi>fo) an d Pn(fi>fo) > w ' tn £ = 0.05. It is noted that 
the power loss in P n (/i,/o) is much smaller than that in P n (A</o) '• Figure 3-15. This is evidence 
of the penalty paid when a false alarm of less than or equal to a is guaranteed at any £ ^T ; 
instead of only at f Q . 
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Figure 3-14: Probability of detection with E = 1, N = 1, a = 10~ 5 , s = 0.01, y = 0.1, and 



N * m = o- 1 - 
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Figure 3-15: Probability of detection with E = 1, N = 1, a = 10~ 5 and e set to 0.05. 
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3.3 Detection when the waveform is known down to an unknown initial phase 

In this case, /i(x n ,y n ) is not induced by a memoryless process, and the theorems- due to 
Huber- used in sections 3.1 and 3.2 cannot be used in the search for the optimal robust decision 
rule. We can, however, use the results from section 3.1 to construct robust decision rules which 
should provide good protection against occasional outliers. In this section we will develop such 
ad-hoc, but intuitively pleasing decision rules, and analyze their performance. For more work on 
robust detection of processes with memory, the reader is referred to [4]. 

We will begin by deriving a Neyman Pearson like rule for the non robust case. The reason for 
doing this will become obvious later on. In section 2.4 we derived our decision rule based on the 

metric log -^- where 

/ho 

f H i(x n ,y n ) = ^j f H1 (x n ,y n \ e = 6)d6 (Eq. 3.3.1) 



/hi 



I III 

In this section, we will examine the Neyman Pearson like rule based on the metric log — 



'mx 



fno 
where 

f Hli9mx = max{f H1 (x n ,y n \ 9): 9} (Eq. 3.3.2) 

and 6 mx is the 6 which maximizes the conditional probability density function. Using the 
notation in section 2.4, we find 



1 - 2 



/^ n '^)=^^exp(- — IT 



^ (Eq. 3.3.3) 

+ n£])exp (— 1/ * i?e{exp(y'[GJ + 0])} ) 
N 
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where T is defined in (Eq. 2.4.16), V is defined in (Eq. 2.4.17), and a> is defined by (Eq. 2.4.18). 
From (Eq. 3.3.3) it is easy to see 6 mx = —a) and 



W^y") = (^rr ex P (~w [T2 + nE])exp ( ^ y) (Eq - 3 - 3 - 4) 



Our Neyman Pearson like rule decides in favor of H t if 



**£--£♦£">' 



Where y is selected to satisfy the false alarm constraint. After simplification, our Neyman 
Pearson like rule reduces down to the exact same rule found in section 2.4. In other words, the 
optimal Neyman Pearson test is equivalent to a test which finds the maximum likelihood 
estimate of 6 and then does the Neyman Pearson detection. Based on this observation we will 
design our robust detection rule as follows, first use robust estimation to estimate 6 and then 
use an appropriate robust decision rule from Section 3.1. 

3.3.1 Robust Estimation of the unknown initial phase 

The maximum likelihood estimate of 6 can be found by forming complex numbers from the 
data pairs z t = x t + jyi and forming complex numbers h t = cos(p(i)) + j sin (p(0), 
computing the complex metric T = Xf=i Zih\ and 8 is the angle of T, 6 = atan2(Im{T}/ 
Re{T}). This is not an M-estimate; however, the real and imaginary components of T are M- 
estimates. We therefore examine the components used in the metric. Let 

R t = -X iS m (p(i)) + Yicos (p(0) (Eq. 3.3.6) 

S t = Xicos (p(0) + Y iS m (p(i)) (Eq. 3.3.7) 
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where all random variables are conditioned on 8. Then clearly 

JRi + > S t (Eq. 3.3.8) 

Since i? ; and S ; are linearly combinations of Gaussian variables, they also are Gaussian. In fact 
{i?J are Independent Identically Distributed Gaussian variables with mean y/~E sin (0 O ) and 
variance N , and {SJ are Independent Identically Distributed Gaussian with mean y/~E cos (0 O ) 
and variance N . We can now develop a robust estimate of 8 , denoted 8* , by independently 
using robust estimates of the means of {i?J and {SJ. 

The formal mathematical approach to finding the robust estimate of a location parameter (the 
mean) of a generating process within a specific class of functions 3 is due to Huber and can be 
found in detail in Detection and Estimation (p320-30) [3]. This work will summarize the results 
and assumptions as they pertain towards robust estimate of the means of Gaussian variables. 

Let us consider the possible occurrence of extreme independent outliers on independent 
identically distributed Gaussian data. Let /i(u) denote the density function of a zero-mean 
Gaussian variable with variance N . Let outliers appear with frequency s and let their 
distribution be represented by some unknown density function g whose entropy and Fisher 

information measure exist, and which is such that g(u) = g(—u) V u; j R g(u)[log(,g(u) + 

H(g)] du < co; g(u) < oo v u, where H(g) is the entropy of g and g(u) = —g(u). Let 3 be 

the class of all valid density functions which can be written 

/(u) = (l-£)A(u) + £0(u) (Eq. 3.3.9) 
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where g can be any density function which satisfies the constraints given above. Given / in 3, 
let 1(f) denote the Fisher information measure. Define /* which if exists satisfies 



/(/ * )= /eV (/) 



(Eq. 3.3.10) 



For < e < 0.03, /* is as follows 



f 1 



/*(*) = 



N n \JN, 



a 



1 I x 



( ca \\ 2 \ / ex \ 
\2NJ\ V \2NJ 



-== Jexpf-—- \x-b\ J 



a < x < a 



a<|x|<6 (Eq. 3.3.11) 



\x\ > b 



Where the constants a, b, c are such that 



ctan 
b \ b 



( ca \ 



01- 



+ S 



a 



cos 



/ ca Y\~ 2 
V2ivJ. 



2c 



ca (ca\ 



0\- 



-l^-e 



(Eq. 3.3.12) 



Let y/ = -/*//*. For x> 0, 



r C ( ex \ 

7i-tan — - 



< x < a 



yf(x) = < 



X 

— - a < x < b 



N, 



o 



N n 



x > b 



(Eq. 3.3.13) 



and y/*(x) = — y/*(—x). Given some data sequence x n generated from some f(u — m), where 
f(u) is within 3 and m is a constant, the robust estimation of m, denoted in is 
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n 

m*(x n ): V y/[x t - m*(x n )] = (Eq. 3.3.14) 

i = l 

This robust estimation m*(x n ) is consistent at every / £ 3. Let /?(^,/) be the rate of 
convergence of some M-estimate y/ at /. Let p be the class of all y/ such that yAjx) = 
— \f/{— x). The rate of convergence of our robust M-estimate is such that 

PW.f) < /?(>*,/*) < Piv.D V/ £ 3, V^£ p (Eq. 3.3.15) 

We define our robust estimate 8 as follows: use the estimation rule in (Eq. 3.3.13) to obtain 
estimates R* and S* and then define 8 = arctan2{R* /S*). At a given / £ 3 as n -» oo the 
random variables 7?* and 5* are asymptotically Gaussian. 

We will consider 

f(x t ,yi) = (1 - £)fi(xi,yi) + £g x , y (xi,yi) (Eq. 3.3.16) 

where g(x;,y;) = g x (xi)g y (yi) and where ^ x (x ; )and g y (y;) are defined via a discrete random 
variable q as follows 



Prob{q = 0} = 0.5 

Pro% = 1} = 0.5 (Eq - 3 - 3 - 17) 



1 /x; — WE cos(p(i) + 8 ) + mcos(p(i))] N 

; q = 



#*(>;) = 



iV V V^o 



i z- , (Eq. 3.3.18) 

1 /xj — \yE cos(p(i) + O ) — mcos(p(i))J\ 

0M ?^f ;<? = 1 



VM 
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g y {yd = 



y t — [VE sin(p(i) + 8 ) + msin(p(j))] 



;q = 



1 /y, - [Vf sin(p(i) + 6 ) - m sin(p(0)]\ 

n 7^ );q = 1 



W 



(Eq. 3.3.19) 



Define / R (u) = (1 — £)/i iR (u) + £g R (u) as the density function of the variable 7?, where R is as 
defined in (Eq. 3.3.6) and {x;,yj are realizations of random variables defined by (Eq. 3.3.16). 
Also define f s (u) = (1 — s)f ls (u) + sg s (u) similarly where S is as defined in (Eq. 3.3.7), we 
obtain 



.5 fu-(VEsm(6 ) + m)\ ,5 
g R (u) = -= (/>[ — ! + 



u — (V£ sin(0 o ) — m) 



IN„ 



(Eq. 3.3.20) 



0.5 (u - (VEcos(8 ) + m) 
9s(u) = -j=0[ 7= 



+ 



.5 (u — (V£cos(0 o ) — m) 



(Eq. 3.3.21) 



Clearly, g R (u — yJ~E sin(0 o )) and g s (u — V£ cos(0 o )) are symmetric density functions around 
zero; the outlier model is such that f R ,f s E 3 where 3 is defined in (Eq. 3.3.9). The robust 
estimates R* and S* at f R and f s are asymptotically Gaussian. Denote [i R and a R as the mean 
and standard deviation of R* at f R as n -* °o. Define fi s and a s similarly. By applying the 
theorem in Detection and Estimation (p 312) [3], we find 



[i R = VE sm(9 ) 



(Eq. 3.3.22) 



fi s = VE cos(6 ) 



(Eq. 3.3.23) 
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or = n 



. Co W^ u ~ HR)] 2 fR(u)du 



^Co !^*(" - r)f R (u)du 



T = ^R 



(Eq. 3.3.24) 



_ 1 Co[^("-M5)] 2 /5(")d« 



ct| = n 



fa Co V (u - t)/ 5 (u) du 



T = /i S 



(Eq. 3.3.25) 



To simplify f_ 00 [yf*(u — fi R )] 2 f R (u)du and J_ oo [i//*(u — [i s )] 2 f s (u)du, we will define a new 
function 



a) t (a, c, N Q ,m) 



a-m 
J-JI^N 2 



tan z 



c {y4~N~o + m ) 



2AL 



^(y)^y (Eq. 3.3.26) 



and we will also use the following equalities 



,oo b 2 x / 



00 b 2 1 (y — m\ b 2 (b — m 



b N 2 ^fN~ \^fN~ J 



Ni 



(Eq. 3.3.27) 



f b y 2 i (y - m\ 

m 2 +N r 



Ni 







b — m\ (a — m 

- 



V^o , ( a ~ m \ V^o r fb — m 



+ — T (a + m)(/> 



N ) Ni 



(b + m)(f) 



(Eq. 3.3.28) 



We obtain 
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.Coiy (« - /%)] 2 /s(")du = L°°J^*(" - H R )] 2 f R (u)du = 2(1 - 
ecola,c,NO,0+£cola,c,NO,m+21-£NoNo2<DbNo-<DaNo+NoNo2a^aNo 
-NoNo2b<f>bNo+£m2+NoNo2®b-mNo-(E>a-mNo+NoNo2a+m<pa- 
mNo-NoNo2b+m(f)b-mNo+21-£b2No20-bNo+£b2No20-b+mNo (Eq. 3.3.29) 



,., 3 f «> „ 



(5 r<x> 



To simplify — J" ^*(u — j)f R (u)du\ and — /_ yf(ji — T)f s (u)du\ , we will define a 



flr-'-oo 



'T = ^R 



flr-'-oo 



new function 



'T = ^s 



o) 2 (.a,c,N ,m) 



u—rn 

[ ^° 
I m 



2N; 



-sec 2 ( zrr. ) <f>(y)dy 



2N, 



o 



(Eq. 3.3.30) 



We obtain 



d 
dr. 



p oo Pi r°° 

y/(u- r)f R (u) du = — y/(u- r)f s (u 



)du 



T=^R 



T = ^s 



= 2(1 — £)a) 2 (<2, c, JV , 0) + ea) 2 (a, c, N ,m) 



+ ■ 



2(1 -e) 



Wn 



ft \ /a 



+ ■ 



Wn 



ft — m\ I a — m 
0\ , 1- 



(Eq. 3.3.31) 



We will now find the probability density function of our estimate 0* as n -* oo. To simplify 
future expressions we will find the probability density function of #, /(#), where i9 = 9* — 6 . 
A realization of i9 is found as the angle of (S* + y'i?*)exp (— ]9 ) . The complex random variable 
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(5* +y'7?*)exp (— ]9 ) is Gaussian with both real and imaginary parts having variance a} = e£ 
and whose real mean is l/if + +/i| = V£ and whose imaginary mean is zero. Define 



z n = /m{(S* + jR*) exp(-j0 o )} (Eq. 3.3.32) 



z d = Re{(S* +jR*)exp(-\9 )} (Eq. 3.3.33) 



where Im{x] and Re{x} denote taking the imaginary and real part of x respectively. Define 



/m{(S*+77?*)exp(-jg )} z n 

Z = — — — = — (Eq. 3.3.34) 

Re{(S*+jR*)exp(-)9 )} z d 



We find 



q + ->C7\ farctan(Z); if z d > 

* = arctanliZ) = (^^ + ^ . f ^ < Q (Eq. 3.3.35) 



Let / z , z + and f z \ z - denote the probability density function of Z conditioned on z d > and 

z d < respectively. Let f$ denote the probability density function of the random variable #. 
We obtain 

( (1 + tan 2 i9)/ z| + (tan(i9))Prob{z d > 0}; - 7 < ^ < 7 
U(fi) = \ n 3n (Eq- 3.3.36) 

(1 + tan 2 0)/ z , z -(tan(0))ProM(z d < 0)}; - < d < — 
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/ z|z + (tan 09))ProMz d > 0} 

1 



ay2n 



cos(i9) exp 



Esin 2 (d) 
2a 2 



r- , fVEcos(-0) 
+ V£cos 2 (i9)<Z> 



o~cos(i9) 



V£cos(i9) 



O" 



(Eq. 3.3.37) 
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1 / Esin 2 (d) 

|cos(i9)| exp 



ay2n 
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er|cos(i9)| ^ 



V£cos(i9) 



r- v , -V£|cos(i9)|\ 
-VE cos 2 (i9)<Z>( I 



O" 



(Eq. 3.3.38) 



Using (Eq. 3.3.37) and (Eq. 3.3.38) we can write (Eq. 3.3.36) as 



A/£cos(i9) N 



VFcos(t9) 



A/EsinG9)\r A/£cos(i9)\ ^ A/£co 
/*(*) = </>{ -^ J ^( ^^) + Vfcos(0)<p( 



(Eq. 3.3.39) 



The probability density function of 9* as n -» oo is 



/s*W =U(.x-9 ) 



(Eq. 3.3.40) 



3.3.2 Robust Detection 

As noted earlier, when the initial phase is unknown (Section 2.4), the optimal detection at the 
nominal model is equivalent to finding the maximum likelihood estimate for 9 . We denote this 
estimate 9 , assuming 9 = 9 and examining the same metric used in optimal detection at the 
nominal model where the waveform is completely known (Section 2.2) . In similar fashion, 
based on our outlier model, we can use the appropriate robust detection metric found in 



sections 3.1.1 or 3.1.2 under the assumption that 9 = 9* , where 9* is obtained from the 
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method described in 3.3.1. We will assume an outlier model where outliers occur per data pairs 
(x;,y;) and therefore, we will adopt the detection rule in section 3.1.1. We will focus our 
analysis on the case where the occurrence of outliers is the same under both H and H 1 and is 
less than 0.5 (s = s = s t < 0.5). Define 



ki(8*) = Xicosipi + 9*) + ytsinipi + 6*) 



(Eq. 3.3.41) 



di 
z(ki) = \ kt 

do 



ki<d 
d-i < k t < d Q 

k t >y[E -d 



(Eq. 3.3.42) 



where d is the solutions to (Eq. 3.1.19). The robust decision rule is 



<r(fc") = 



1L 

1 k n :n- i y z (k i )>A* 

i = l 
n 

r* k n :n- i y z (k i ) = A* 

i = l 
n 

k n :n- 1 Yz(k i )<A* 



(Eq. 3.3.43) 



where z(/q) is as in (Eq. 3.3.42) and where A* and r* are such that 



Pr In- 1 V z(/q) > A* f * ! + r* Pr In' 1 Y z(/q) = 



fo\ = a 



(Eq. 3.3.44) 



The least favorable density functions / * and /j* are given by (Eq. 3.1.16) and (Eq. 3.1.17). The 
density function ofn~ 1 'Zi=iZ(k i ), conditioned on / * and some 8* , is the same regardless of 
the value of 6*. Therefore the density function of n _1 Zf=i^(^i) conditioned on / * is the same 
as the density function of n~ 1 'Zf=iZ(k i ) conditioned on / * and 9* = 6 . From section 3.1.1, 
this density function is asymptotically Gaussian with mean and variance given by (Eq. 3.1.23) 
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and (Eq. 3.1.24), respectively. Denote by rrif and Vf respectively the mean and variance of 
ft -1 Zf=i Z(.ki) conditioned on / *. 

For asymptotically large values of n, the constant r* in (Eq. 3.3.43) can be taken equal to zero; 
we thus then obtain 

X* = m f5 + v f5 0~ 1 {l-a) (Eq. 3.3.45) 

Denote by m^(0*) and vj (6*) respectively the mean and variance of the random variable 
ft -1 Zf=i z(Ki)> conditioned on f t and 9* , for large n values. Since {^} conditioned on f t and 
6* are independent identically distributed Gaussian variables with mean Videos (6* — 8 ) and 
variance N , m^(0*) and vj (0*) are respectively equal to m gm (pi) and vj m (/t) in (Eq. 3.1.29) 

and (Eq. 3.1.30) with fi = -jLcos(*P - 6 ). Let 

/ = (1 - £ )A + £# m (Eq. 3.3.46) 



where 



.5 (k-UEcos(6* -6 ) + mcos(6*)) 



N n V JW, 



.5 /fc-(VIcos(0*-0 o )-mcos(0*))\ (Eq. 3.3.47) 



N V V^o 



Equivalently, g m (k) is the density resulting from the density functions g x .(Xi) an d SV (ji) ' n 

i y i 

(Eq. 3.3.18) and (Eq. 3.3.19). Define 



1 (k - (VI cos(0* - 9 ) + mcos(9*))\ 
g ma (k) =—</>[ 5: ^ V ^-Jl\ ( E q. 3.3.48) 



N V JN 
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1 (k - (V£ cos(0* - 6 ) - mcos(B*))\ 

g mb (k) =~f=n " " 7=* —^ ) (Eq- 3-3.49) 



"o 



We rewrite (Eq. 3.3.46) as, 



/ = (1 - c)A + |5ma + |5mB (Eq. 3.3.50) 



Denote by m 5ma (0*) and Vg (6*) respectively the mean and variance of the random variable 
n~ 1 Yii=iz(.Ki), conditioned on g ma and 6* , for large values of n . Define m flmfc (0*) and 
Vg .0*) similarly. Since g ma (k) is a Gaussian probability density function with variance N , 
m 9ma^ and v 9ma(P) are respectively equal to m g Jji) and v^Jji) in (Eq. 3.1.29) and (Eq. 
3.1.30), with pi = -j=cos(9* — O ) + mcos(6*\ Similarly, since g m b(k~) is a Gaussian 
probability density function with variance N , ?% mfc (0*) and Vg . (j@*) are respectively equal to 
m gm (jl) and v^ m Ql) in (Eq. 3.1.29) and (Eq. 3.1.30), with y. = -j=cos(6* - 9 ) - mcos(6*). 

Denote by m.f(6*) and vj(6*} respectively the mean and variance of the random variable 
ft -1 Hf=i z(.Ki), conditioned on / in (Eq. 3.3.50) and 6* , for large values of n . We obtain 

m f (p) = (1 - e)m /l (^) + \m gr JP) + Hm^) (Eq. 3.3.51) 
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£ „ ,_ N £ 



V/ 2 0*) = a - £ )^(e*) + - v 2 gmb {e*) + - v 2 gmb {e*) 



+ n~ 1 



{i- £ )Um h {e*)-m gma {e*)] 



+ (l-e) £ -(m fl (e*)-m gmb (e*)y 



(Eq. 3.3.52) 



e 2 



^(^W-^Wj 



Let us denote by P n p(f) the power induced by the robust decision rule in (Eq. 3.3.43) at the 
density function / conditioned on 9*. 



Due to (Eq. 3.3.45), we conclude: for n -* oo, 



f m f (B*) - m f * - v f * 1 (1 - a) N 
Pn,Hf) = <P[ -^ f -^f " ) (Eq. 3.3.53) 



Let us denote by P n (f) the power induced by the robust decision rule in (Eq. 3.3.43) at the 
density function / in (Eq. 3.3.50). From our analysis it is difficult to find P n (f), since 8* is directly 
computed from our observed data (x",y"). If 6* was completely independent of the data 
sequence (x n ,y n ) and still had the distribution given by (Eq. 3.3.40), the theorem of total 
probability would give: 



P n (f\9* selected independent of x n ,y n ) 

rn+d 
J-n+Br. 



n+e ° (Eq. 3.3.54) 



where fg*(x) is defined in (Eq. 3.3.40). Asn -» oo any single data pair (X(,y ; ) contribution to 8* 
is negligible; therefore the distribution of z(Ki) conditioned on a 6* selected using the data pair 
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(Xi,yi), is close to the distribution of z(7^) conditioned on a 8* selected independently of 
(x£, yi). This logic cannot be extended to n _1 £f =1 z(K{). 

3.3.3 Comparing robust detection with non-robust detection 

Without a straight forward way to compute P n (f), even asymptotically, we will compare the 
performance of the robust and the optimal detectors, both under the assumption that the angle 
estimates, 8* and 8, were selected independently of the data sequence (x",y"). One way to 
achieve this independence is to assume that the analyst receives two separate independent 
sequences of data pairs from the same generating process, each of length n. The analyst uses 
the first to estimate 6* or 8, and then based on this estimate uses the second to decide H 1 or 
H . We conjecture that this comparison will be at least qualitatively similar to the comparison of 
the case where the angle estimates are made directly from the same data sequence used to 
decide H 1 or H . 

The 8* and 8 estimates can be expressed as 



8 = arctanl \ * ) (Eq. 3.3.55) 



\n 






Ir = arctanl \ ™ | (Eq. 3.3.56) 



Denote by fi(rri) the distribution defined in (Eq. 3.3.16). We note that /i* (0) = f t , the nominal 
distribution. Let us denote by 8(m) the probability distribution of 8. Define 8*(m) similarly. 
Under /i and / x * , as n -> oo, the variables -2f=i^i , -£f=i-!>i <-Zf=i^i< and -£?=i-!>i are 
Gaussian; therefore, the 6(m), and 6*(m) are of the mathematical form found in (Eq. 3.3.40). 
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We find that 8(m), and 6*(m) both have the same value of E but generally different a. Let us 
denote by a(m) and a*(m) the a parameter of 6(m) and 6*(m) respectively. Figure 3-16, 
shows a(m) and a*(m) for various values of m where N = 10, n = 100, and s = 0.001. 
When m = 0, a(m) < a*(m) shows that the optimal estimate at f t is superior. However, as m 
increases, a(m) increases without bound, whereas a*(m) does not; 6*(m) does not change 
significantly as a function of m. Figure 3-17 shows cr(0) and er*(0) as a function of s at f x where 
N = 10, and n = 100; as s increases cr*(0) increases showing the tradeoff between protecting 
against a greater frequency of outliers and optimal performance at f t . Figure 3-18, shows 0(0) 
and #*(0) where E = 1, N = 10, s = 0.03 andn = 100; the closeness of the two distributions 
indicates that the performance of robust estimate is close to the performance of the optimal 
estimate at f x . Figure 3-19, shows 0(100) and 0*(1OO) where E = 1, N = 10, e = 0.03 and 
n = 100; the difference in the two distributions shows the danger of using the optimal estimate 
at f t with outliers present. 
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0.320 



0.318 
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Figure 3-16: Sigma comparison where N = 10, n = 100, and s = 0.001. 
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Figure 3-17: Sigma comparison at f t where JV = 10, and n = 100. 
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Figure 3-18: Probability density functions at f x where E — 1, N Q = 10, s = 0.03 and 

n = 100. 
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Figure 3-19: Probability density functions at ft with m = 100, where E = 1, N = 10, 

e = 0.03 and n = 100. 

Denote by P* (/ *, m) the power of the robust decision rule obtained by independently 
estimating 9* and then using the robust decision rule given by (Eq. 3.3.43) at the density pair 
[foifi ( m )]i P*{fo> m ) is found using the integral found in (Eq. 3.3.54). Denote by P(f ,rn) the 
power of the robust decision rule obtained by independently estimating § and then using the 
robust decision rule given by (Eq. 2.2.1) at the density pair [f ,ft(jn)\, P{.f > m ) is found using 
an integral with the same form as (Eq. 3.3.54), where the robust conditional power and robust 
probability density replace their the non-robust equivalents. Figure 3-20 shows P*(/ *,m) and 
P(f , m) where m = 0, E = 1, N = 10, e = 0.03 and a = le — 5 as a function of n. The 
difference between the two curves shows the performance loss induced by the robust decision 
rule designed at the highest permissible frequency of outliers (s = 0.03) as compared to the 
optimal decision rule, both in the absence of outliers. Figure 3-21 shows P*(/ *,m) and 
P(f , rri) as functions of , where m = 100, E = 1, N = 10, e = 0.03 and a = le — 5 . The 
difference between the two curves shows the performance loss from using the optimal decision 
rule in the presence of outliers as compared to robust decision rule. 



87 




400 600 

n 



800 



Figure 3-20: Power curves at / t *(0) = A where E = 1, N = 10, e = 0.03 and 

a = le — 5. 
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Figure 3-21: Power curves at /i*(100) where E = 1, N = 10, e = 0.03 and 

a — le — 5. 
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4 Thesis Conclusions 
4.1 Summary of Findings 

In Section 1, we developed the Neyman-Pearson detection rules for an FSK modulated signal 
that is transmitted via a channel modeled by additive white Gaussian noise and is filtered by an 
ideal brick-wall filter, for three cases: 1) when the waveform is known completely (coherent 
detection), 2) when the waveform is known except for a random initial phase (incoherent 
detection) and 3) when only the energy of the waveform is known (energy detection). To start, 
we used only first and second order statistics to compare the three detectors. Then, we 
compared the three detectors using full statistical descriptions of the involved waveforms. 
While the first and second order statistical analysis led to simple and often quoted performance 
comparison figures, its results were expectedly highly inferior to those induced by the full 
statistical analysis. The full statistical analysis showed that the performance of the three 
detectors is generally closer than the indications induced by the first and second order statistics 
analysis. 

In Section 2, we developed the robust detectors for the three cases cited above, where the 
classes for both the null and emphatic hypothesis are the Huber classes. For the coherent 
detection case, we considered two outlier models: 1) when outliers occur over complex data 
pairs and 2) when outliers occur independently per datum. We derived analytic expressions for 
their asymptotic performance, when the outliers are drawn from a Gaussian distribution. Finally, 
for various models, we compared their performance to that of the optimal at the nominal 
distribution pair (the non-robust detector) detector, as the latter was developed in the previous 
section. For the energy detection case, we similarly derived the robust detector and compared it 
to the non-robust detector found in the previous section. In general, in the absence of outliers, 
our analysis showed slightly reduced performance for the robust detectors, as compared to that 



89 

of the non-robust detectors. In contrast, in the presence of outliers, our analysis showed 
significantly reduced performance for the non-robust detectors, as compared to that of the 
robust detectors. Both the coherent and the energy detection generating processes considered 
were memoryless; as a result, the robust detection schemes were developed using existing 
theorems and are solutions to saddle-point optimization problems. However, the incoherent 
detection generating process is not memoryless and no known theorems could be used to 
derive the optimal detection scheme. Despite this, we were able to develop an intuitively 
pleasing incoherent robust detector for the incoherent case. Our robust detector first used the 
data to derive a robust initial phase estimate; using the latter phase estimate, our detector 
subsequently used the robust detection established methods for the coherent case. Due to the 
inter-dependence between the phase estimate and the subsequent detection, we did not find 
analytic expressions for the asymptotic performance of our detector. We derived analytic 
expressions for the asymptotic performance of a slightly modified incoherent robust detector, 
however, which removed the inter-dependence between the phase estimate and the detection 
steps. We compared the latter analytic expressions to non-robust versions, in both the presence 
and the absence of outliers. In the absence of outliers, these comparisons showed a slight 
performance loss when the robust detector is used, as compared to the case when the non- 
robust detector is used, instead. In the presence of outliers, the robust detector significantly 
outperformed the non-robust detector. 

While our analysis focused specifically on the detection of FSK signals, Section 1 highlighted the 
importance of using full statistical models, when different detectors are compared and; Section 
3 highlighted the danger of relaying on non-robust detectors in the presence of outliers. Non- 
robust detectors relay on complete knowledge of the underlying data generating distributions. 
In general, for all real-world problems, full knowledge of the data generating processes is non- 
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feasible. Our analysis may derive the generating distributions from approximate physics-based 
models. Otherwise, our analysis may attempt to estimate these distributions using data 
observations. In either case, the generating distributions are only approximately known. For the 
latter case, in particular, the tails of the distributions are difficult to estimate correctly. Robust 
detection performs well in the presence of outliers which occur at extreme values represented 
by the tails of our nominal distribution. In contrast, non-robust detection performance breaks 
down in the presence of even infrequent outliers. We therefore conclude that robust detectors 
should always be used over non-robust detectors, for all real-world problems. 

4.2 Future Research 

There are several areas to expand and further the research presented in this thesis. Future work 
could examine the non-robust and robust detection schemes of non FSK signals. While much of 
the results in this thesis extend to non-FSK band-limited signals with constant energy, many of 
the results need to be modified to address signals with time-varying energy such as Amplitude 
Modulated signals. In addition, this thesis examined detection under the assumption of an ideal 
brick-wall filter; future work could examine the detection schemes of signals using other filter 
models. Of particular interest are filters which are found in receiver hardware such as Finite 
Impulse Response filters. The current thesis focused on Neyman Pearson detection schemes 
where the number of samples for a given detection scheme is fixed. Future work could examine 
the performance of Sequential Detection Testing for both the non-robust and robust cases. The 
robust detection schemes performed data truncation based on the frequency of outliers. For 
most real-world applications, the frequency of outliers will not be known a priori, and may not 
be constant. Future work could examine methods to optimally estimate this frequency, and 
update this estimate as needed. Future work could also examine performance of the various 
detection schemes on simulated and real data. This performance could be compared to the 
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analytic performance presented in this thesis. Some of the performance curves found in this 
thesis were based on the assumption of a large number of samples. Future work could do 
performance analysis assuming a small number of samples. While this thesis examined 
detection of FSK signals, future work could examine the optimal schemes to demodulate FSK 
signals with particular interest in robust demodulation algorithms. Finally, future research could 
examine ways to make the FSK signals more robust to extreme outliers, thereby increasing 
performance of robust detection demodulation algorithms. 
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Appendix A : Extension of Huber Robust Detection 

Let 3% and 3% be two disjoint classes of n-dimensional density functions, and let 3% and 3% 
represent respectively hypothesis H 1 and H . Denote by {/™:/i,i, ■■■,fi, n } a set of density 
functions within the class 3? . Assume that x™ is generated by a single set of density functions in 
either 3% or 3%, denoted f™ . Finally, assume that 3% and 3^ contain only memoryless 
processes. Then, given n, an observation vector x n , and some /" we have 



1L 

fi(x n ) = Y\fiM) 



7 = 1 



(Eq.A.l) 



Given some pair (/ n ,/i n ) , our non-robust decision rule is 



S(x n ) = 



1 x tl :rW=n- 1 flogS)>r 
fri \fo,i(.Xi)J 

* x n :nx n ) = n- i y\og(^p^)>A* 
fri \fo,i( x iV 

x" : T(x n ) = n 1 Y log f ^4^1 > ^ 
I £? VoMV 



(Eq.A.2) 



where r*, 1* are selected to satisfy the false alarm constraint. If some pair (/ * n , /i* n ) exists, with 
decision rule <5*(x") such that 



P(«5,/i* n ) < P(8*jr) < P(8*,f?) VA n £ 3f 



(Eq.A.3) 



and 



P(5*,/o n ) < P(8*,f *) = a V/ n £ 3£ 



(Eq. A.4) 



then 5*(x n ) is the robust decision rule. 



94 



Lemma A.l: Let there exist a least favorable pair (f*^, f£j) for every i: 1 ... n, under a single 
sample hypothesis test, let 3% and 3% contain only memoryless processes and let each least 
favorable pair satisfy 



^(fg \<y 



f A>Vr\iog\ f j^\<y 

fu 



f*. 

J 0,1 






(Eq.A.5) 



E3%,Vyintf 



fi,i\ Vi:l n,Vf u £^,V/ 0|i 



Then, the pair (/ *f, / x *f ) is the least favorable density pair over J^ and 3% . In other words, if a 
process is memoryless, the least favorable density pair can be found by taking the least 
favorable density pair over each data sample, conditioned on each least favorable density pair 
satisfying (Eq. A.5). 

Proof: The condition in (Eq. A.5) implies that, given f oi £ 3 . and f ti £ 3 ., a random variable 

7 and non-decreasing functions ^ ;(y),^* ; (y),^ ; (y) and g i; (y) exist, such that g ,i(y) ^ 

9*o,i(y) ^ gl,i(y) ^ 9i,i(y) V y in m, and the functions 9 ,iiy),9o,$y)>9\,$y)> 0i,i(y) 
coincide respectively with the probabilities found in (Eq. A.5). The proof then follows the proof 
found in Detection and Estimation (Lemma 6.3.1, p. 172-3) [3], replacing instances of 

g iy),g*oiy),giiy), #1 (y) with g ,i(y),g*o,i{y),gl,i{y), g^iy)- 

If 3 i and 3 t ; are defined as Huber's classes, where Huber's classes are defined as 



5 ,i = [ftOd = (1 - E )f 0ii (x) + E h(x) h £ | 



(Eq.A.6) 
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5i,i = ifito = (1 - eO/uGO + sMx) hE% (Eq. A.7) 

where E, is the class of all one-dimensional density functions within 91, then the least favorable 
density pair is 



flito = 



(l-e )/o,iW 



Xm fo,ito ° li 



c 1 (l-e )/i #t U) x: - > c 0ii 

fo.iW 



(Eq.A.8) 



A'iW = 



(l-£i)/i,iW 



x: foM u 



c x (1 - £i)/ ,j (x) x: J 1,1 < c ,j 

Jo,iW 



(Eq.A.9) 



where c oi , c ti are such that 






/o,i +Co]Pr 






> 



Co, 



/i.t( = (l-eo) _1 (Eq.A.10) 






A, ll + c U Pril|g, S c,, 



/o,i ( = (l-ei)- 1 (Eq.A.ll) 



If we consider a robust detection rule over 3% and 3% , where 3^i and 3^i are the Huber's 
classes defined by (Eq. A.6) and (Eq. A.7) and where 3% and 3^ contain only memoryless 
processes, then, due to Lemma A.l and (Eq. A.8) and (Eq. A.9), our robust detection rule is 



1 x n \T*(x n )>X* 

S'(x n ) = \r* x n :T*(x n ) = A* 
x n :T*{x n ) < I* 



(Eq.A.12) 
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where r* , A* are selected to satisfy the false alarm constraint, and where 



1L 

T*(x n ) = n- 1 Y j YUxd (Eq.A.13) 

£=1 



C/uGO 

log[c 1; i(l - £i)(l - £ ) _1 ] x:f u {x)/f oA (x) < c u (Eq. A.14) 

log[(l - £i)(l - £o) _1 /i,iW//o,iW] x: c u < /i,i(x)// 0|i W < c 0ii 
log[c 0; i(l - £0(1 - £ ) _1 ] x:f u (x)/f 0ii (x) > c 0ii 



